Stream: wg-async-foundations

Topic: futures crate(s)


nikomatsakis (Jun 06 2019 at 12:25, on Zulip):

Hey @boats, @Taylor Cramer --

One of the questions that I don't have a good handle on is the futures crate. Looking at the rust-lang-nursery/futures-rs crate, I see a lot of "subcrates". Is there some consensus about the relative importance of these various features? Is the plan to move some/all of this into libstd at some point, once it's had more time to bake?

boats (Jun 06 2019 at 15:10, on Zulip):

There's not a clear consensus. My understanding of the current plan is that some time between Future being stabilized and async/await being stabilized, we will release an 0.3 of the crates which will be substantially similar to the current API.

boats (Jun 06 2019 at 15:10, on Zulip):

We've discussed a 1.0 version at the all hands, but there were several outstanding issues that made that seem more difficult to produce

boats (Jun 06 2019 at 15:11, on Zulip):

In the long term, there's a general sense that we'd like core abstractions like stream, asyncread, asyncwrite to be in std, but its still very unclear and up in the air

Florian Gilcher (Jun 06 2019 at 15:22, on Zulip):

Would it be possible to extract an uncontroversial subset and release that as 1.0?

boats (Jun 06 2019 at 15:28, on Zulip):

I wanted to look into doing that but I think it will have to be delayed until after async/await is stable

Taylor Cramer (Jun 06 2019 at 17:19, on Zulip):

@Florian Gilcher unfortunately the uncontroversial subset does not include Stream

Taylor Cramer (Jun 06 2019 at 17:20, on Zulip):

AsyncRead/AsyncWrite IMO are the most "obvious"ly correct, but even they have points of disagreement that folks have argued over.

Taylor Cramer (Jun 06 2019 at 17:21, on Zulip):

futures-core and futures-ioare meant to represent a subset that is stable enough for folks to rely on in public API boundaries

Taylor Cramer (Jun 06 2019 at 17:22, on Zulip):

everything else is intended to be left as utilities/implementation details that can evolve freely over time as they grow to better fit the needs of users.

centril (Jun 06 2019 at 18:05, on Zulip):

I would be a concerned with a development of having too many AsyncX traits

centril (Jun 06 2019 at 18:05, on Zulip):

that would presumably cause a lot of code duplication and be unfortunate from an effect system POV

Taylor Cramer (Jun 06 2019 at 18:26, on Zulip):

@centril I don't think that's avoidable

centril (Jun 06 2019 at 18:26, on Zulip):

elaborate?

Taylor Cramer (Jun 06 2019 at 18:26, on Zulip):

Like, this is the "what color is your function" problem in its purest form

centril (Jun 06 2019 at 18:26, on Zulip):

This problem has a known solution: Monads or effect systems.

centril (Jun 06 2019 at 18:27, on Zulip):

We cannot do the former; we can do the latter.

Taylor Cramer (Jun 06 2019 at 18:27, on Zulip):

Neither of those solve this problem, and there's tons of existing literature into why

Taylor Cramer (Jun 06 2019 at 18:28, on Zulip):

but I'd be happy to discuss w/ you why and what an "async trait"-like effects system might look like, and why I think it's unlikely to pan out, at least anytime in the near future

centril (Jun 06 2019 at 18:28, on Zulip):

Not the literature I've read (e.g. Frank, Idris's algebraic effects, etc.) but I'm happy to discuss it

Taylor Cramer (Jun 06 2019 at 18:28, on Zulip):

I'm not completely defeatist on the idea, but I think it's very far from where we are today

centril (Jun 06 2019 at 18:29, on Zulip):

@Taylor Cramer aside: up for pre-triage mtg?

Taylor Cramer (Jun 06 2019 at 18:33, on Zulip):

@centril yeah, it doesn't look like I'm getting kicked out of my room yet

Taylor Cramer (Jun 06 2019 at 18:33, on Zulip):

often I can't get a room :(

centril (Jun 06 2019 at 18:33, on Zulip):

neat

Florian Gilcher (Jun 06 2019 at 18:45, on Zulip):

@Taylor Cramer I don't see why we absolutely need Stream, TBH

Florian Gilcher (Jun 06 2019 at 18:45, on Zulip):

From my point of view, Stream as a stable and all-encompassing abstraction takes time, but for any subset of users needs, either using a concrete channel implementation or an ad-hoc library trait will do.

Taylor Cramer (Jun 06 2019 at 19:03, on Zulip):

@Florian Gilcher interesting. I think it's necessary for the same reasons that Iterator is necessary, but maybe you're considering a more specific use-case than the ones I have in mind

Matthias247 (Jun 07 2019 at 03:53, on Zulip):

I think I already tried to start a few discussions around this which never got traction. But here’s my opinion again:

I think neither Streams nor the AsyncXyz things should be stabilized right now, and until someone can paint a clear path forward for a reasonable integration with futures. Up to now only Nemo157 could do that partially, and even they weren’t sure if it would really work out. Up to that point even the current prominence of those types in crates like Runtime feels unfortunate. I wouldn’t want to see a 3 color function world in Rust. And neither people building on top of abstractions which get canned later on.

Florian Gilcher (Jun 07 2019 at 08:45, on Zulip):

@Matthias247 how do you want to stop people from building on top?

Florian Gilcher (Jun 07 2019 at 08:46, on Zulip):

Like, honestly, people are waiting for this for years now, they will either take what we give or build their own.

Florian Gilcher (Jun 07 2019 at 08:47, on Zulip):

@Taylor Cramer you mean as adapters? I don't think they fill the same niche as Iterator, especially not in a sense that they need to be available as a finished interface from the get-go

Florian Gilcher (Jun 07 2019 at 09:05, on Zulip):

Like, I appreciate that they should be present, but useful applications of considerable size can be written without, which gives a bit of leeway to not stabilising them.

boats (Jun 07 2019 at 10:55, on Zulip):

@Florian Gilcher no one wants to include these things in the MVP

boats (Jun 07 2019 at 10:58, on Zulip):

I mean I guess youre talking about leaving stream out of a 1.0 version of the futures crate

Matthias247 (Jun 07 2019 at 16:12, on Zulip):

@Florian Gilcher You can’t stop but you can discourage. E.g. by leavings things out of stabilization as proposed. And by at least communicating that this is an open area, and investing some work in it (experimenting and then proposing a path forward). Right now the official communication sounds like: Stream is the way to go, and we might make it somehow work with Generators. Due to that projects like runtime and go are picking things up as their core abstraction.

Taylor Cramer (Jun 07 2019 at 18:23, on Zulip):

I don't know what a 1.0 version of the futures crate would include if not Stream. It seems like it'd just be what's in std, which is the only "true" 1.0 in my mind

Florian Gilcher (Jun 09 2019 at 16:30, on Zulip):

@boats my initial question was if there's a subset of the futures crate that _is_ ready for 1.0 and it has slowly drifted that people seem to feel like such a subset is not useful.

Florian Gilcher (Jun 09 2019 at 16:31, on Zulip):

@Taylor Cramer futures combinators for example, potentially asyncread/write to build protocols on a more abstract fashion then reading sockets directly.

stjepang (Jun 10 2019 at 14:05, on Zulip):

I wonder if AsyncRead/AsyncWrite/Stream should use proper async fn foo instead of fn poll_foo (once async trait methods become possible). What do you think about renaming current AsyncRead/AsyncWrite to PollRead/PollWrite instead?

stjepang (Jun 10 2019 at 14:05, on Zulip):

So we'd have something like:

trait PollRead {
    fn poll_read(
        self: Pin<&mut Self>,
        cx: &mut Context,
        buf: &mut [u8]
    ) -> Poll<Result<usize, Error>>;
}
stjepang (Jun 10 2019 at 14:05, on Zulip):

Then, once async trait methods become possible, we can add:

trait AsyncRead: PollRead {
    async fn read(buf: &mut [u8]) -> Poll<Result<usize, Error>>;
}
stjepang (Jun 10 2019 at 14:06, on Zulip):

Finally, we deprecate polling traits in favor of the new traits using async fn and add the following blanket impl.

impl<T: PollRead> AsyncRead for T {
    async fn read(buf: &mut [u8]) -> Result<usize, Error> {
        future::poll_fn(|cx| self.poll_read(cx, buf)).await
    }
}

This way, libraries using PollRead can switch to AsyncRead without breaking backwards compatibility.

stjepang (Jun 10 2019 at 14:08, on Zulip):

The problem with poll_foo methods is that they don't return futures. It is (in a philosophical sense) wrong for File to have poll_read() method because, well, files are not futures. However, operations on files (like read or write) are futures.

stjepang (Jun 10 2019 at 14:10, on Zulip):

This especially becomes a problem when asynchronous operations need to keep some state. For example, with poll_read, async read operations cannot keep their own state. And if there is any state inside an async read operation, then it has to be embedded into the File struct.

stjepang (Jun 10 2019 at 14:12, on Zulip):

But even that's fraught with problems because then there is no way to "cancel" async operations with poll_read(). If you begin a read operation by calling poll_read() and then timeout on it, you can't drop the future to cancel it because there isn't one! That means the next read operation will try to pick up the state of the previous poll_read() invocation, which isn't right.

stjepang (Jun 10 2019 at 14:14, on Zulip):

All those problems make me believe poll_foo methods should only belong to real futures (i.e. async operations), and not to objects that can be operated on asynchronously (i.e. start async operations).

Nemo157 (Jun 10 2019 at 17:56, on Zulip):

@stjepang main discussion of that idea is at https://github.com/rust-lang-nursery/futures-rs/issues/1365

Nemo157 (Jun 10 2019 at 17:59, on Zulip):

the major issue that cramertj mentions is that async fn read in a trait is actually sugar for an associated type + function (well, almost certainly going to be, once it's defined), once you have that associated type on the trait how do you deal with dyn AsyncRead?

Taylor Cramer (Jun 10 2019 at 18:36, on Zulip):

the alternative solution is to have it be a dynamically-dispatched future of some kind, but it's unclear exactly how that would work

Taylor Cramer (Jun 10 2019 at 18:37, on Zulip):

one proposal that i'd been playing around with was a kind of automatic version of what we did for Waker, which is to offer by-value DSTs with a fixed "max size" (in the case of waker, the size is "one pointer width")

Taylor Cramer (Jun 10 2019 at 18:38, on Zulip):

we could do something sorta clever like making the object returned the size of all the arguments to the function + alignment so that you could store them all in a struct and return that type to prevent having to heap-allocate for all the existing impls while retaining object safety

Taylor Cramer (Jun 10 2019 at 18:38, on Zulip):

it's really complicated though and requires a whole field of language functionality we don't have today

Taylor Cramer (Jun 10 2019 at 18:41, on Zulip):

e.g.:

trait AsyncRead {
    async fn read(&mut self, buf: &mut [u8]) -> Result<usize, io::Error>;
}

would desugar into something like this:

trait AsyncRead {
    fn read<'a>(&'a mut self, buf: &'a mut [u8]) -> dyn<sizeof(usize) * 2> Future<Output = Result<...>> + 'a;
}
Taylor Cramer (Jun 10 2019 at 18:42, on Zulip):

where dyn<NUM> Trait is some by-value dyn trait with an upper size bound of NUM

Taylor Cramer (Jun 10 2019 at 18:42, on Zulip):

and implementing it with an async fn whose body was too big would require writing async(Box:new) or something

stjepang (Jun 10 2019 at 18:50, on Zulip):

the major issue that cramertj mentions is that async fn read in a trait is actually sugar for an associated type + function (well, almost certainly going to be, once it's defined), once you have that associated type on the trait how do you deal with dyn AsyncRead?

Do we need dyn AsyncRead? Can we just make traits with async fns not object-safe?

Taylor Cramer (Jun 10 2019 at 18:53, on Zulip):

@stjepang yes, dyn AsyncRead is critically important

Taylor Cramer (Jun 10 2019 at 18:55, on Zulip):

can we make traits with async fns not object-safe?

Yes, but people will continue to hack around them and not use them if that's the case. I'm not opposed to starting here necessarily, but it's not a generally applicable solution.

Jake Goulding (Jun 10 2019 at 19:02, on Zulip):

dyn AsyncRead is critically important

As a curious bystander, why is that?

Taylor Cramer (Jun 10 2019 at 19:07, on Zulip):

@Jake Goulding it's a core point over which libraries and other tools allow abstraction of underlying interfaces (notably TCP), and making all of the relevant functions generic over those parameters is either not an option in some places due to heterogeneous collections, or it would cause an incredibly large amount of code bloat

Taylor Cramer (Jun 10 2019 at 19:08, on Zulip):

(note that all the same comments apply to Stream, if you viewed it as async fn next(&mut self) -> Option<Self::Item>;)

Jake Goulding (Jun 10 2019 at 19:09, on Zulip):

Why is this code bloat more of a problem compared to plain-old Read or Iterator (which I use as my sync version of Stream)?

stjepang (Jun 10 2019 at 19:16, on Zulip):

@Taylor Cramer I'm curious too - I've never seen dyn Iterator, dyn Read, or dyn Write before. Are async versions really used with dynamic dispatch?

Taylor Cramer (Jun 10 2019 at 19:17, on Zulip):

I've used all of those a bunch

Taylor Cramer (Jun 10 2019 at 19:17, on Zulip):

@Jake Goulding it isn't, but the same comments apply to Read and Iterator

Taylor Cramer (Jun 10 2019 at 19:18, on Zulip):

those are used w/ dynamic dispatch very often

Jake Goulding (Jun 10 2019 at 19:38, on Zulip):

Hmm. We must experience drastically different types of code, I suppose; I find dynamic dispatch relatively uncommon.

Jake Goulding (Jun 10 2019 at 19:39, on Zulip):

However, it wouldn't surprise me if most functions took the generic and then certain classes of callers used the trait object

Taylor Cramer (Jun 10 2019 at 19:42, on Zulip):

a search for &mut Read on github turns up a few thousand results, but I don't know how large that is by percentage

Taylor Cramer (Jun 10 2019 at 19:42, on Zulip):

https://github.com/search?q=%22%26mut+Read%22+language%3ARust&type=Code

Taylor Cramer (Jun 10 2019 at 19:43, on Zulip):

Box<Read> + Box<dyn Read> is only about 200

Taylor Cramer (Jun 10 2019 at 19:43, on Zulip):

Box<Iterator also gives a few thousand

Taylor Cramer (Jun 10 2019 at 19:44, on Zulip):

&mut Iterator is ~2k

Jake Goulding (Jun 10 2019 at 20:48, on Zulip):

We encourage people to take the generics (e.g. https://rust-lang-nursery.github.io/api-guidelines/interoperability.html#generic-readerwriter-functions-take-r-read-and-w-write-by-value-c-rw-value) because it works for both cases.

Taylor Cramer (Jun 10 2019 at 20:49, on Zulip):

it also generates a heck of a lot more code ;)

Taylor Cramer (Jun 10 2019 at 20:49, on Zulip):

and doesn't work for heterogeneous collections

simulacrum (Jun 10 2019 at 20:50, on Zulip):

hm, it should work with heterogeneous collections?

Taylor Cramer (Jun 10 2019 at 20:50, on Zulip):

I've personally found trait objects to be extremely useful and would be concerned if we didn't have a story for them

Taylor Cramer (Jun 10 2019 at 20:50, on Zulip):

@simulacrum perhaps what I said was unclear: e.g. HashMap<u32, Box<dyn Read>> won't work with generics

Jake Goulding (Jun 10 2019 at 20:51, on Zulip):

I dunno about "a heck of a lot" because I simply haven't done that investigation; can you point to any info I can read to have a sense of the magnitude?

Taylor Cramer (Jun 10 2019 at 20:51, on Zulip):

it only works due to trait objects

Jake Goulding (Jun 10 2019 at 20:51, on Zulip):

Mostly I know about the format machinery, where the codegen was indeed too much

Taylor Cramer (Jun 10 2019 at 20:51, on Zulip):

@Jake Goulding I've personally changed a few functions in fuchsia to use trait objects rather than generics and dropped compilation times and binary size by more than 30%

simulacrum (Jun 10 2019 at 20:52, on Zulip):

Box<dyn Read> implements Read though so I'd sort of expect it to work...?

Taylor Cramer (Jun 10 2019 at 20:52, on Zulip):

as far as more holistic samples, I know @Alex Crichton has done some in the past

Taylor Cramer (Jun 10 2019 at 20:52, on Zulip):

@simulacrum to be clear, the Box<dyn Read> is exactly the piece I'm talking about

Taylor Cramer (Jun 10 2019 at 20:52, on Zulip):

you can't write HashMap<u32, Box<impl Read>>

Taylor Cramer (Jun 10 2019 at 20:52, on Zulip):

and store multiple different Read types

simulacrum (Jun 10 2019 at 20:53, on Zulip):

oh, yes, that's true -- but if you have R: Read you can always Box it (though you might end up with multiple levels of boxes, which is unfortunate)

Taylor Cramer (Jun 10 2019 at 20:53, on Zulip):

Sure-- the point is that if AsyncRead wasn't object safe, then things like that wouldn't be possible

Jake Goulding (Jun 10 2019 at 20:53, on Zulip):

@Taylor Cramer How did that affect runtime performance? I don't know much about Fuchsia's target machines, are they binary-size limited?

simulacrum (Jun 10 2019 at 20:53, on Zulip):

Ah, yes, that's true

simulacrum (Jun 10 2019 at 20:54, on Zulip):

Though you can sometimes work around it (e.g., create a "fake" object like we do with RawWaker IIRC)

Jake Goulding (Jun 10 2019 at 20:54, on Zulip):

And to be clear, I agree that AsyncRead will ideally be object-safe

Taylor Cramer (Jun 10 2019 at 20:54, on Zulip):

@Jake Goulding

I don't know much about Fuchsia's target machines, are they binary-size limited?

That's not something I can address publicly

Jake Goulding (Jun 10 2019 at 20:54, on Zulip):

I was mostly investigating the "critically important" modifier

Taylor Cramer (Jun 10 2019 at 20:54, on Zulip):

Binary size is important

Jake Goulding (Jun 10 2019 at 20:55, on Zulip):

In general sure, cause of cache sizes and all that.

simulacrum (Jun 10 2019 at 20:55, on Zulip):

I'd expect that for most things were AsyncRead is worthwhile the async-ness is slow enough that the speed gained by generics would probably not be worth it? i.e., network is too slow

Taylor Cramer (Jun 10 2019 at 20:55, on Zulip):

Like I said, I can't speak more specifically

Jake Goulding (Jun 10 2019 at 20:55, on Zulip):

I didn't realize that part wasn't public, I apologize for asking.

Taylor Cramer (Jun 10 2019 at 20:55, on Zulip):

no worries! it's an annoying limitation of the world at present :)

Jake Goulding (Jun 10 2019 at 20:56, on Zulip):

@simulacrum an interesting point about the expected usecases of the two traits.

Taylor Cramer (Jun 10 2019 at 20:56, on Zulip):

I'd expect that for most things were AsyncRead is worthwhile the async-ness is slow enough that the speed gained by generics would probably not be worth it? i.e., network is too slow

somewhat, yes, although note that some of these usecases are things like "pipe the data coming from a local socket to stdout"

simulacrum (Jun 10 2019 at 20:57, on Zulip):

stdout and such are likely either file i/o or terminal i/o or something like that I suspect so that seems like it'd just support the "don't use generics" argument

Taylor Cramer (Jun 10 2019 at 20:58, on Zulip):

at the end of the day I haven't seen very many if any AsyncRead impls that would benefit greatly from being written with async/await such that they would outweigh the rest of these tradeoffs

Taylor Cramer (Jun 10 2019 at 20:58, on Zulip):

not including the limitations around what we can actually do with such a type in terms of storing the output future (which is not only nameless, but contains a lifetime parameter that can't be abstracted over due to current limitations of HRTB / GATs)

Taylor Cramer (Jun 10 2019 at 20:59, on Zulip):

and I doubt that many people would implement AsyncRead for their own custom types without the help of combinators

Taylor Cramer (Jun 10 2019 at 20:59, on Zulip):

(it's easy to allow async { ... } implementations of AsyncRead today with an intermediate combinator type)

Taylor Cramer (Jun 10 2019 at 21:00, on Zulip):

@simulacrum yup, they don't need to be crazy fast, but "fast enough" to be useful for things like file transfers

simulacrum (Jun 10 2019 at 21:01, on Zulip):

sure, where the box probably won't matter I'd guess? i.e., LLVM still sees and optimizes both sides

Taylor Cramer (Jun 10 2019 at 21:01, on Zulip):

mhm

simulacrum (Jun 10 2019 at 21:01, on Zulip):

the only problem would be non-buffered file io, but that's slow regardless

Taylor Cramer (Jun 10 2019 at 21:02, on Zulip):

yeah, we have AsyncBufReader and AsyncBufWriter for that

simulacrum (Jun 10 2019 at 21:02, on Zulip):

My one gripe/concern with Box<dyn Trait> is that you want to be sure to use it everywhere

simulacrum (Jun 10 2019 at 21:03, on Zulip):

e.g. you don't want to have crate A which uses it, B which has T: Trait and C which has Box<dyn Trait> in a callchain since you'll end up with Box<Box<dyn Trait>> at the end

simulacrum (Jun 10 2019 at 21:03, on Zulip):

which is... two levels of pointer chasing?

Taylor Cramer (Jun 10 2019 at 21:18, on Zulip):

yeah, it'd be great to have specialization around to help solve that one

simulacrum (Jun 10 2019 at 21:20, on Zulip):

(you sort of want HKT here I think since ideally you'd also not constrain to Box and accept anything "small" but not have to double box and such)

simulacrum (Jun 10 2019 at 21:20, on Zulip):

though I'm not sure if HKT is the right thing, I'd need to do some more research to be sure.

Taylor Cramer (Jun 10 2019 at 21:45, on Zulip):

@Jake Goulding

How did that affect runtime performance?

I realized I didn't address this part of your question-- it didn't at all, but that was particular to the specific place in which the change was made.

Taylor Cramer (Jun 10 2019 at 21:45, on Zulip):

Other parts of the codebase certainly rely on generics to avoid multiple levels of Boxing

Taylor Cramer (Jun 10 2019 at 21:45, on Zulip):

which would be performance-critical

simulacrum (Jun 10 2019 at 21:46, on Zulip):

fwiw you can actually get around the double boxing bit on stable today if your type is 'static via Any

Matthias247 (Jun 11 2019 at 03:35, on Zulip):

Good discussion! I think I might give a +1 to the idea of @stjepang . The naming at least indicates that those are not "real" async types, but all handwritten ones.

I agree that dynamic dispatch would be great to have! I don't have any performance concerns, since I've never experienced any important impact in the past (e.g. comparing boost asio with boxed types vs. templatized ones). The main concern I have is overhead due to dynamic allocations, but trait objects don't necessarily require dynamic allocations.

However if I have to decide between the "being able use trait objects" and "being able to implement types via async functions" I think I would lean towards the latter.

Matthias247 (Jun 11 2019 at 03:40, on Zulip):

I don't agree that one ever wants to implement AsyncRead (or: async byte streams in general) only manually and never through composition.
E.g. in my HTTP/2 library for .NET the Stream implementation (HTTP stream) conforms to a byte stream interface which is basically the equivalent of AsyncRead + AsyncWrite. Reading and writing data is purely implemented in terms of async functions that use async primitives (mutexes, semaphores, internal streams): https://github.com/Matthias247/http2dotnet/blob/master/Http2/StreamImpl.cs#L371-L480

That worked out pretty great, and I wouldn't want to write those things all in a manual fashion.

Matthias247 (Jun 11 2019 at 03:44, on Zulip):

Or as an easier example: The Implementation of an async data pipe, which implements AsyncRead on one end and AsyncWrite on the other. This can be composed on top of a ringbuffer, a synchronous mutex and two async ManualResetEvents and a two async fns. There is no need to manually touch or store a Waker and to manually implement poll()s

Matthias247 (Jun 11 2019 at 03:57, on Zulip):

@Taylor Cramer

we could do something sorta clever like making the object returned the size of all the arguments to the function + alignment so that you could store them all in a struct and return that type to prevent having to heap-allocate for all the existing impls while retaining object safety

I like the idea, but I think it can't be done based on the argument alone. The size of the returned Future is determined also be how much state will be on the "stack" of the async fn. And therefore an implementation which uses more "stack" won't fit in the preallocated space for a smaller object. we can probably determine some upper bounds through educated guesses and uses those. Then the functions might return something along FixedBox<Size=256>(dyn Future). The downside is obviously that large futures won't fit, and for small ones we are wasting space.

Maybe the compiler feature that would help on this is some kind of whole program analysis that inserts at compile-time the actually used number of bytes based on the biggest implementation of the trait that it encountered?

We could also use a box which only allocates dynamically when a given size treshold was exceeded (like https://docs.rs/smallbox/0.7.3/smallbox/index.html), but that only seems to make the interface even more restricted and obscure while not solving the general case.

nikomatsakis (Jun 11 2019 at 16:59, on Zulip):

the major issue that cramertj mentions is that async fn read in a trait is actually sugar for an associated type + function (well, almost certainly going to be, once it's defined), once you have that associated type on the trait how do you deal with dyn AsyncRead?

it feels like we need to solve this regardless

Florian Gilcher (Jun 13 2019 at 12:57, on Zulip):

In general, I agree with @Taylor Cramer that trait objects are often underused, especially at system boundaries.

Florian Gilcher (Jun 13 2019 at 15:05, on Zulip):

There's another interesting issue that came up yesterday: There's a pretty trivial implementation of Read/Write for AsyncRead/AsyncWrite in futures compat, but it relies on the context being task local and retrievable via Current: https://rust-lang-nursery.github.io/futures-api-docs/0.3.0-alpha.9/futures/compat/struct.Compat.html#impl-Read

Florian Gilcher (Jun 13 2019 at 15:05, on Zulip):

Now, that implementation is quite useful, e.g. to bridge the gap to native-tls and such. Is there a good way to implement this using the new futures?

simulacrum (Jun 13 2019 at 15:10, on Zulip):

Personally I think we need to think about the implicit contexts via compiler/language mechanism, not TLS, that @eddyb and a few others have talked about -- it'd be useful for this sort of "I need global-ish state in a bunch of functions" type thing, such as TyCtxt and Session in the compiler formatting infrastructure etc

simulacrum (Jun 13 2019 at 15:10, on Zulip):

I think I've heard that some language has this? Maybe .. Racket, but not sure

Florian Gilcher (Jun 13 2019 at 15:55, on Zulip):

Not sure, but I'd actually be interested, as someone who works in fields where "per request" context is a big thing.

Florian Gilcher (Jun 13 2019 at 15:56, on Zulip):

That's long-term, though, but the question was more if there's a halfway elegant solution with what we have.

simulacrum (Jun 13 2019 at 16:03, on Zulip):

https://github.com/PistonDevelopers/dyon/issues/224 is basically what I was talking about but I think that's inspired by some other language

eddyb (Jun 13 2019 at 20:06, on Zulip):

@simulacrum my ideas are mostly that of eternaleye, FWIW, so you might want to get a hold of him (are you on IRC? Freenode, specifically)

eddyb (Jun 13 2019 at 20:06, on Zulip):

I could try to get him in here, hmpf

simulacrum (Jun 13 2019 at 20:06, on Zulip):

Not currently but can be in a week or so (currently in the process of moving, so servers/internet are a bit up in the air)

Matthias247 (Jun 14 2019 at 03:34, on Zulip):

@simulacrum I think you might be referring to something like implicits in Scala or a language-assisted dependency-injection mechanism in general. For Kotlin this one was recently proposed: https://github.com/Kotlin/KEEP/blob/46e6a347432d868635e9c6161ca332581315b185/proposals/compile-time-dependency-resolution.md

Dependency injection mechanisms are definitely helpful to cleanup the amount of require boilerplate. However I think of those as mostly a helpful thing and not a must-have. People should be able to write the same code by manually threading through all required arguments too, which involves less magic and has less compatibility issues.

simulacrum (Jun 14 2019 at 13:56, on Zulip):

@Matthias247 I don't think that's quite the right feature -- and sometimes you can't ergonomically thread the argument through into e.g. a fmt::Display/fmt::Debug impl

simulacrum (Jun 14 2019 at 13:58, on Zulip):

and AIUI this feature would have less overhead than e.g. storing the value in structs since we'd not be using stack space for it -- the compiler would point at some top-level stack frame with the value where it knows a pointer to it or the value itself is

Nemo157 (Jun 14 2019 at 16:09, on Zulip):

@Florian Gilcher that actually sounds similar to what https://www.reddit.com/r/rust/comments/c08178/openssl_async_adapter/ is using

Nemo157 (Jun 14 2019 at 16:10, on Zulip):

But via explicit adaptor fields rather than implicit context

Last update: Nov 18 2019 at 01:30UTC