Stream: t-libs/wg-allocators

Topic: Design of `AbortAlloc` not compatible with `try_reserve`


Tim Diekmann (Nov 21 2019 at 18:54, on Zulip):

AbortAlloc is implemented as a wrapper around an arbitrary allocator A. AbortAlloc maps the returend error to handle_alloc_error. So if an OOM occures, there is no way to recover. Thus, AbortAlloc is incompatible with try_reserve. Any thoughts/idea on this?

Erich Gubler (Nov 22 2019 at 00:28, on Zulip):

What do you mean that it's incompatible?

Tim Diekmann (Nov 22 2019 at 06:28, on Zulip):

When calling try_reserve (or any other try_method), you expect to get an allocation error on OOM. AbortAlloc aborts before the function returns.

Tim Diekmann (Nov 22 2019 at 06:28, on Zulip):

This is the desired effect on methods like reserve, but not on try_resersve.

gnzlbg (Nov 28 2019 at 17:01, on Zulip):

Is try_reserve stable ?

gnzlbg (Nov 28 2019 at 17:01, on Zulip):

I think I would prefer to, e.g., get try_ versions of all allocating methods, not only reserve, but all of them, by just using a PanicAlloc that panics, such that the panic can be caught

John Ericson (Nov 28 2019 at 17:01, on Zulip):

I think it is stable

gnzlbg (Nov 28 2019 at 17:02, on Zulip):

The default allocator can abort on OOM

John Ericson (Nov 28 2019 at 17:02, on Zulip):

*unstable

John Ericson (Nov 28 2019 at 17:02, on Zulip):

I tried to prevent it from being stabilized for exactly this reason

gnzlbg (Nov 28 2019 at 17:02, on Zulip):

We can always deprecate it afterwards

gnzlbg (Nov 28 2019 at 17:12, on Zulip):

One situation were this approach wouldn't work is if -C panic=abort.

gnzlbg (Nov 28 2019 at 17:13, on Zulip):

In that case, Vec::try_reserve would still work properly, since no panics occur, but a PanicOnOomAlloc<MyAlloc> adapter would terminate the process.

John Ericson (Nov 28 2019 at 17:14, on Zulip):

@gnzlbg in my PR, I used repr(transparent) so I could cast on and off the wrapper

John Ericson (Nov 28 2019 at 17:15, on Zulip):

So if you want today's try_reserve, you just cast away the PanicOnOomAlloc<_>, and then call try_reserve.

John Ericson (Nov 28 2019 at 17:16, on Zulip):

It's slightly more verbose but I don't really care. I think anyone that's using try_reserve today is much better off without PanicOnOomAlloc anyways.

John Ericson (Nov 28 2019 at 17:17, on Zulip):

try_reserve is a hacky way to make the thing you actually want to do hopefully succeed. But that's racy; you still want to do not panic on the thing itself. And in that case unless you are reserving way before, you probably don't need the try_reserve at all.

gnzlbg (Nov 28 2019 at 17:20, on Zulip):

It's slightly more verbose but I don't really care. I think anyone that's using try_reserve today is much better off without PanicOnOomAlloc anyways.

@John Ericson without or with ?

John Ericson (Nov 28 2019 at 17:20, on Zulip):

without

gnzlbg (Nov 28 2019 at 17:20, on Zulip):

So you are in favour of only doing try_reserve ?

John Ericson (Nov 28 2019 at 17:20, on Zulip):

@gnzlbg if you scroll up in that issue you posted in, you can read me describe the stuff I am saying here with more words

gnzlbg (Nov 28 2019 at 17:20, on Zulip):

ok i will

John Ericson (Nov 28 2019 at 17:21, on Zulip):

@gnzlbg I mean that PanicOnOomAlloc definitely should exist, but you shouldn't be mixing sometimes panicking and somtimes not panicking

Simon Sapin (Nov 28 2019 at 17:21, on Zulip):

The default allocator can abort on OOM

No, in today’s standard library it’s Vec and other callers that check for errors and call handle_alloc_error which in turn defaults to abort.

gnzlbg (Nov 28 2019 at 17:23, on Zulip):

@John Ericson this one: https://github.com/rust-lang/rust/issues/48043#issuecomment-501416660 ?

gnzlbg (Nov 28 2019 at 17:23, on Zulip):

Im not sure I fully understand what you are propossing, it feels as if you want all Vec methods to return a Result

Simon Sapin (Nov 28 2019 at 17:24, on Zulip):

AbortAlloc is implemented as a wrapper around an arbitrary allocator A. AbortAlloc maps the returend error to handle_alloc_error. So if an OOM occures, there is no way to recover. Thus, AbortAlloc is incompatible with try_reserve. Any thoughts/idea on this?

Yes, making the choice to abort or not be part of the type signature is incompatible with having it only as part of the choice of method being called on a given type. I personally prefer the latter.

gnzlbg (Nov 28 2019 at 17:24, on Zulip):

No, in today’s standard library it’s Vec and other callers that check for errors and call handle_alloc_error which in turn defaults to abort.

@Simon Sapin yes, but I can provide a #[global_allocator] that aborts on OOM or panics instead by using AbortOnOOM<std::alloc::Heap>; right ?

John Ericson (Nov 28 2019 at 17:26, on Zulip):

I think https://github.com/rust-lang/rust/issues/48043#issuecomment-409967652 is the first in my thread. It's somewhat hard to follow the convo rereading it again I'll admit. These sorts of discussions tended to be strewn around a bunch of issues, sadly.

John Ericson (Nov 28 2019 at 17:26, on Zulip):

@gnzlbg yes there should be a try_ version of every method that returns a result.

gnzlbg (Nov 28 2019 at 17:27, on Zulip):

Ah gotcha

Simon Sapin (Nov 28 2019 at 17:27, on Zulip):

There’s nothing technically stopping you from aborting inside your #[global_allocator], but https://doc.rust-lang.org/std/alloc/trait.GlobalAlloc.html#errors “encourages” not to

gnzlbg (Nov 28 2019 at 17:27, on Zulip):

So do you want try_clone as well in the Clone trait ?

John Ericson (Nov 28 2019 at 17:27, on Zulip):

Probably yes

gnzlbg (Nov 28 2019 at 17:28, on Zulip):

Like if I extend a Vec<T>, and try_reserve succeeded, but extending requires calls to T::clone, and those T::clone allocate, then, you are screwed

John Ericson (Nov 28 2019 at 17:28, on Zulip):

exactly!

Simon Sapin (Nov 28 2019 at 17:28, on Zulip):

I think the idea of an AbortOnOOM wrapper type is flawed

gnzlbg (Nov 28 2019 at 17:28, on Zulip):

What's the goal of AbortOnOOM ?

John Ericson (Nov 28 2019 at 17:28, on Zulip):

the only way to systematically ensure your program will not crash is to never panic and ? the Result --- ideomatic error handling.

Simon Sapin (Nov 28 2019 at 17:29, on Zulip):

I don’t know, you mentioned it

gnzlbg (Nov 28 2019 at 17:29, on Zulip):

I mentioned PanicOnOOM

gnzlbg (Nov 28 2019 at 17:29, on Zulip):

which is quite differently

Simon Sapin (Nov 28 2019 at 17:29, on Zulip):

https://rust-lang.zulipchat.com/#narrow/stream/197181-t-libs.2Fwg-allocators/topic/Design.20of.20.60AbortAlloc.60.20not.20compatible.20with.20.60try_reserve.60/near/182124881

gnzlbg (Nov 28 2019 at 17:29, on Zulip):

I was asking if that is a possibility

John Ericson (Nov 28 2019 at 17:30, on Zulip):

the non-try methods would require the allocator parameter to have ! as the error type, which means that alllocation failure is "impossible", which means something panics. PanicOnOOM would have the ! error type.

gnzlbg (Nov 28 2019 at 17:30, on Zulip):

and if your goal is to abort on any OOM, it is quite simple

gnzlbg (Nov 28 2019 at 17:30, on Zulip):

and doesn't rely on anybody calling the handle_alloc_error

Simon Sapin (Nov 28 2019 at 17:30, on Zulip):

It’s technically possible. IMO it’s not a good idea.

gnzlbg (Nov 28 2019 at 17:30, on Zulip):

the same applies if you want to panic on any OOM

gnzlbg (Nov 28 2019 at 17:30, on Zulip):

Why?

Simon Sapin (Nov 28 2019 at 17:31, on Zulip):

Why would you panic or abort if the caller has specifically written code to handle both outcomes of try_reserve?

Simon Sapin (Nov 28 2019 at 17:31, on Zulip):

That just seems hostile to me

gnzlbg (Nov 28 2019 at 17:32, on Zulip):

Because if I'm writing #[global_allocator], i'm in charge of building the final binary, and I can do whatever I want?

John Ericson (Nov 28 2019 at 17:32, on Zulip):

@gnzlbg or you can read https://github.com/rust-lang/rust/pull/60703 where I implemented everything I've talked about

gnzlbg (Nov 28 2019 at 17:32, on Zulip):

Like, maybe I just want my binary to fail as fast as possible on OOM without any kind of cleanup - its kind of up to me

John Ericson (Nov 28 2019 at 17:32, on Zulip):

@Simon Sapin with the associated error type try_* would return Result<_, !>, with PanicOnOOM so the user gets a heads up that their code isn't doing what they through via dead code warnings.

gnzlbg (Nov 28 2019 at 17:33, on Zulip):

One way to implement all standard collections more "concisely" would be to wrap the user allocator in a HandleAllocError<Alloc> wrapper internally, that always calls handle_alloc_error on OOM.

Simon Sapin (Nov 28 2019 at 17:33, on Zulip):

@John Ericson I’m sorry, you and I already had this discussion a couple times and I don’t have anything new to add

John Ericson (Nov 28 2019 at 17:34, on Zulip):

The things you are telling @gnzlbg seem like the same things you have told me?

gnzlbg (Nov 28 2019 at 17:34, on Zulip):

@John Ericson I think the difference is that I'm not suggesting for everything to return Result

John Ericson (Nov 28 2019 at 17:34, on Zulip):

I'll not @ you then and at @ @gnzlbg instead, I guess

Simon Sapin (Nov 28 2019 at 17:34, on Zulip):

I meant about Result<_, !> everywhere

John Ericson (Nov 28 2019 at 17:35, on Zulip):

@gnzlbg so without a bunch of new try_ methods, what are you proposing for writing code that never blows up on allocation failure?

gnzlbg (Nov 28 2019 at 17:35, on Zulip):

Vec<T, PanicOnOOM<Alloc>>

John Ericson (Nov 28 2019 at 17:35, on Zulip):

that still panics?

gnzlbg (Nov 28 2019 at 17:35, on Zulip):

it immediately panics, always

John Ericson (Nov 28 2019 at 17:35, on Zulip):

is the application supposed to catch the panic?

gnzlbg (Nov 28 2019 at 17:36, on Zulip):

Yes, it could, if it wanted

John Ericson (Nov 28 2019 at 17:36, on Zulip):

Say you don't want to panic or abort

gnzlbg (Nov 28 2019 at 17:36, on Zulip):

Only works with unwinding

gnzlbg (Nov 28 2019 at 17:36, on Zulip):

If you don't have unwinding, or don't want to unwind, this won't work for you

John Ericson (Nov 28 2019 at 17:36, on Zulip):

that's why I want try_*

John Ericson (Nov 28 2019 at 17:37, on Zulip):

I want my code to not fail, and so I don't write any panics, and I cannot unwind so I do panic == abort

gnzlbg (Nov 28 2019 at 17:37, on Zulip):

But if you want to support Vec<T> failing to resize, T::clone failing, etc. , a global allocator that panics on OOM, and an allocator wrapper that panics on OOM, give you that if you have unwinding

gnzlbg (Nov 28 2019 at 17:37, on Zulip):

And it gives you that without having Result everywhere

John Ericson (Nov 28 2019 at 17:37, on Zulip):

Isn't Result everywhere the ideomatic way of doing error handling in rust?

John Ericson (Nov 28 2019 at 17:37, on Zulip):

and catching panics not?

gnzlbg (Nov 28 2019 at 17:37, on Zulip):

Not for allocation errors

John Ericson (Nov 28 2019 at 17:38, on Zulip):

why is allocation different?

gnzlbg (Nov 28 2019 at 17:38, on Zulip):

Because somebody decided so

gnzlbg (Nov 28 2019 at 17:38, on Zulip):

That decision happened long time ago, and we can't change it

John Ericson (Nov 28 2019 at 17:38, on Zulip):

Right we have some legacy cruft, but I rather things be different for principled reasons.

gnzlbg (Nov 28 2019 at 17:38, on Zulip):

You can write RFCs for adding a lot of try_methods, and maybe they'll go through

gnzlbg (Nov 28 2019 at 17:39, on Zulip):

But for me, unwinding is ok, and a small Alloc<A> wrapper makes allocation fallible, without having to change anything or add any APIs

gnzlbg (Nov 28 2019 at 17:39, on Zulip):

Or rely on people actually using those APIs

John Ericson (Nov 28 2019 at 17:39, on Zulip):

@gnzlbg so what makes this difficult is to me PanicOnOOM, Result, and try_* all go together.

John Ericson (Nov 28 2019 at 17:39, on Zulip):

I need all 3 of them to make the coherent argument

gnzlbg (Nov 28 2019 at 17:40, on Zulip):

I don't need Result and try_

John Ericson (Nov 28 2019 at 17:40, on Zulip):

What does PanicOnOOM do without try_*?

gnzlbg (Nov 28 2019 at 17:40, on Zulip):

because I have unwinding, and can use it to handle OOM

John Ericson (Nov 28 2019 at 17:40, on Zulip):

Say you diddn't use PanicOnOOM. What would happen?

gnzlbg (Nov 28 2019 at 17:40, on Zulip):

PanicOom<A: Alloc>: Alloc is an allocator wrapper, that forwards arguments and returns from / to A, but if a return is a null pointer due to an allocation error, it immediately panics.

John Ericson (Nov 28 2019 at 17:41, on Zulip):

So what happens if I try to do Vec<A> without Vec<PanicOnOOM<A>

John Ericson (Nov 28 2019 at 17:41, on Zulip):

and A returns a null pointer

gnzlbg (Nov 28 2019 at 17:41, on Zulip):

If A forwards to the #[global_allocator] then that would panic, if you make it a PanicOnOOM

gnzlbg (Nov 28 2019 at 17:42, on Zulip):

If A is its own thing, it would call handle_alloc_err at some point, which you can configure to also panic

John Ericson (Nov 28 2019 at 17:42, on Zulip):

panic or abort, right?

gnzlbg (Nov 28 2019 at 17:43, on Zulip):

but then you are relying on Vec always calling handle_alloc_err when it should

gnzlbg (Nov 28 2019 at 17:43, on Zulip):

You can provide your own handler

gnzlbg (Nov 28 2019 at 17:43, on Zulip):

So in my case, I would provide a PanicOnOom global allocator, a handler that panics, etc.

John Ericson (Nov 28 2019 at 17:43, on Zulip):

@gnzlbg I don't feel very motivated by your PanicOnOOM

John Ericson (Nov 28 2019 at 17:43, on Zulip):

So it always panics

John Ericson (Nov 28 2019 at 17:43, on Zulip):

and we just muddle who has the oblilgation

gnzlbg (Nov 28 2019 at 17:43, on Zulip):

yes, and I can catch those panics, if I care about that, and where I care about that

John Ericson (Nov 28 2019 at 17:43, on Zulip):

it would be nice to say "Vec, you don't need to worry about null pointers because that is already taken care of"

John Ericson (Nov 28 2019 at 17:44, on Zulip):

but you cannot rely on the allocator using PanicOnOOM so Vec has to be defensive anyways

John Ericson (Nov 28 2019 at 17:44, on Zulip):

so you get both sides panicking

John Ericson (Nov 28 2019 at 17:44, on Zulip):

what's the point?

gnzlbg (Nov 28 2019 at 17:44, on Zulip):

That there are collections that are not Vec that I also use

gnzlbg (Nov 28 2019 at 17:44, on Zulip):

inside the 100s of my crates I depend on

gnzlbg (Nov 28 2019 at 17:44, on Zulip):

and I don't trustthem

John Ericson (Nov 28 2019 at 17:44, on Zulip):

But they either need to return Result, or panic themselves

John Ericson (Nov 28 2019 at 17:45, on Zulip):

the type system forces this

John Ericson (Nov 28 2019 at 17:45, on Zulip):

Are you worried about those other collections just not handling null pointers at all?

John Ericson (Nov 28 2019 at 17:46, on Zulip):

Then the solution is to make sure that no collection needs to worry about nullness

John Ericson (Nov 28 2019 at 17:46, on Zulip):

@gnzlbg here's the thing, even if every collection is to be used with OOM panics and unwinding, when building collections it's much easier to use Result

John Ericson (Nov 28 2019 at 17:47, on Zulip):

You don't have to be a careful C programmer and remember all your null pointers

John Ericson (Nov 28 2019 at 17:47, on Zulip):

and collections can be made out of other collections, etc etc

John Ericson (Nov 28 2019 at 17:47, on Zulip):

therefore I don't recognize there being a clear division of labor between "collections code" and "applications code"

John Ericson (Nov 28 2019 at 17:48, on Zulip):

It's best if we just add all the try_ methods and associated error type so we can safely create all these other collections, and the responsibility for error handling is clearly laid out and no one needs to be defense without help from the compiler. Nobody is being forced to use the try_* methods, remember.

Tim Diekmann (Nov 28 2019 at 19:01, on Zulip):

I think the idea of an AbortOnOOM wrapper type is flawed

Yes it is, that's the reason why I removed it from alloc-wg. The thing is, that it will always aborts (or panics in the case of PanicOnOom) as soon as an allocation fails, even before try_reserve can return a result.

Personally I'm not fine with relying on unwinding on OOM. That defeats the purpose on returning a Result. Personally I also think, that the current design of the collections isn't that great, but we can't change this anyway, so we shouldn't bother.

John Ericson (Nov 28 2019 at 19:06, on Zulip):

@Tim Diekmann I would be happy to contribute my PR to that library

John Ericson (Nov 28 2019 at 19:07, on Zulip):

I don't get all this "ship has sailed" talk when there is no problem with adding a bunch of new methods

John Ericson (Nov 28 2019 at 19:08, on Zulip):

And there could be some lang thing down the road for Result<T,!> to T

John Ericson (Nov 28 2019 at 19:08, on Zulip):

So it's not like 2x methods is necessary a permanent state of affairs

Tim Diekmann (Nov 28 2019 at 19:09, on Zulip):

Sure, go ahead and let's see, how it turns out. The crate tries to collect proposals of the WG to try them, not to be merged as the new liballoc.

John Ericson (Nov 28 2019 at 19:09, on Zulip):

Ok thanks

Tim Diekmann (Nov 28 2019 at 19:09, on Zulip):

I don't get all this "ship has sailed" talk

Me neither

John Ericson (Nov 28 2019 at 19:13, on Zulip):

@Tim Diekmann now I'm confused :) Didn't you just say "but we can't change this anyway"? Or were you referring to something more specific?

Tim Diekmann (Nov 28 2019 at 19:13, on Zulip):

A few commits back you can see the design with AbortAlloc and the bound for AllocRed<Error = !> for infallible methods .

Tim Diekmann (Nov 28 2019 at 19:14, on Zulip):

Wait a second searching for a cite

John Ericson (Nov 28 2019 at 19:14, on Zulip):

OK

Tim Diekmann (Nov 28 2019 at 19:20, on Zulip):

Can't find it right now. However I don't see why the collections defaults to abort instead of panicking. Or even don't return Result

John Ericson (Nov 28 2019 at 19:20, on Zulip):

OK, sweet!

John Ericson (Nov 28 2019 at 19:23, on Zulip):

@Tim Diekmann oh, looks like alloc-wg does a lot of this stuff already?! Sorry, I am comming back to these things after quite a while

Tim Diekmann (Nov 28 2019 at 19:51, on Zulip):

No problem :D

Tim Diekmann (Nov 28 2019 at 19:52, on Zulip):

As mentioned before: I tried to collect many proposals so it turns out how much sense everything makes.

gnzlbg (Nov 29 2019 at 11:55, on Zulip):

Yes it is, that's the reason why I removed it from alloc-wg. The thing is, that it will always aborts (or panics in the case of PanicOnOom) as soon as an allocation fails, even before try_reserve can return a result.

That is kind of the point right? What can you do to prevent this from happening? e.g. I can write an AbortOnOom wrapper that sets the #[global_allocator] to abort on OOM, and there is nothing the standard library can do against that.

gnzlbg (Nov 29 2019 at 11:58, on Zulip):

For something like PanicOnOom, we can protect against this by saying that a GlobalAlloc that unwinds is UB, and e.g. making the GlobalAlloc hooks nounwind, to be able to optimize under this assumption.

gnzlbg (Nov 29 2019 at 11:59, on Zulip):

But if you write a library that uses Vec::try_reserve, you are not guaranteed a Result on allocation error, because your allocator might abort the process if that happens, and there is nothing you can do about that.

gnzlbg (Nov 29 2019 at 12:00, on Zulip):

On Linux, you might get even an Ok, and only when you touch the memory, the OOM-killer kills your process

gnzlbg (Nov 29 2019 at 12:01, on Zulip):

So the best that Vec::try_reserve can do is guarantee that handle_alloc_error won't be called

gnzlbg (Nov 29 2019 at 12:02, on Zulip):

which is very different from the guarantee that allocation errors will be reported as Err, there is no way to guarantee that in Rust

gnzlbg (Nov 29 2019 at 12:02, on Zulip):

you need the allocator and the OS to conspire together with Vec::try_reserve to support those semantics

gnzlbg (Nov 29 2019 at 12:04, on Zulip):

(the goal of Vec::try_reserve is to support supporting those semantics when all those components align)

Tim Diekmann (Nov 29 2019 at 12:10, on Zulip):

That is kind of the point right?

I tried to modify the collection API, that allocating methods like Vec::reserve can only be used for infallible allocators (AllocRef<Error = !>). To maintain backwards combatibility, I introduced the wrapping allocator AbortAlloc.

Tim Diekmann (Nov 29 2019 at 12:11, on Zulip):

To achieve this, the design of AbortAlloc simply didn't work out

gnzlbg (Nov 29 2019 at 12:11, on Zulip):

Ah yes, that won't work

gnzlbg (Nov 29 2019 at 12:12, on Zulip):

What I think could be done is provide a different FallibleVec in libstd, where all allocating methods return Result<(), AllocRef::Error> or some other error type.

Tim Diekmann (Nov 29 2019 at 12:12, on Zulip):

This brings me to two points:
- Does an infallible generic allocator even makes sense?
- I think this could only be solved with an additional trait: InfallibleAlloc (and InfallibleRealloc?)

gnzlbg (Nov 29 2019 at 12:12, on Zulip):

Then, Vec can be implemented on top of such a FallibleVec type by just .unwrap()ing everything

Tim Diekmann (Nov 29 2019 at 12:13, on Zulip):

What I think could be done is provide a different FallibleVec in libstd

Right, this would be another possibility

gnzlbg (Nov 29 2019 at 12:13, on Zulip):

If we ever get a prelude::v2 in a future edition, we could add a Vec type alias for FallibleVec

gnzlbg (Nov 29 2019 at 12:13, on Zulip):

and offer the old Vec through a std::v1::collections::Vec.

gnzlbg (Nov 29 2019 at 12:13, on Zulip):

Only for crates on the new edition, on the old edition, the prelude would just point to v1, so that would be a backward compatible change.

gnzlbg (Nov 29 2019 at 12:14, on Zulip):

Code in the new edition interfacing with APIs from crates using the old edition using Vec, would need to use std::v1::collection::Vec instead, so this might end up being quite a pain, but... is doable.

Tim Diekmann (Nov 29 2019 at 12:14, on Zulip):

I think introducing FallibleVec wouldn't be the optimal solution, as Vec isn't the only collection which allocates. We would need a fallible variant for all collections

gnzlbg (Nov 29 2019 at 12:14, on Zulip):

Yes

gnzlbg (Nov 29 2019 at 12:14, on Zulip):

but we can do that one collection at a time

gnzlbg (Nov 29 2019 at 12:15, on Zulip):

without breaking backward compatibility, and just changing what the defaults are between editions, while providing both version on all editions

Tim Diekmann (Nov 29 2019 at 12:15, on Zulip):

What do you think on introducing liballoc::alloc::Infallible?

Tim Diekmann (Nov 29 2019 at 12:15, on Zulip):

as trait

gnzlbg (Nov 29 2019 at 12:17, on Zulip):

What does that mean ?

gnzlbg (Nov 29 2019 at 12:17, on Zulip):

An alloc that impls Infallible never fails ?

Tim Diekmann (Nov 29 2019 at 12:17, on Zulip):

Kind of, an alloc, which will never return on OOM

Tim Diekmann (Nov 29 2019 at 12:19, on Zulip):

But I don't know if we end up with the same problem...

Tim Diekmann (Nov 29 2019 at 12:21, on Zulip):

We shouldn't forget that most of the users will never use or write a custom allocator or even use try_reserve.

Tim Diekmann (Nov 29 2019 at 12:22, on Zulip):

So we shouldn't clutter the namespace with too much types.

Tim Diekmann (Nov 29 2019 at 12:25, on Zulip):

But the question remains: How much sense does a generic infallible allocator make? Won't all allocator OOM at a time without introducing additional constraints?

gnzlbg (Nov 29 2019 at 15:38, on Zulip):

I think that until somebody actually tries to implement this, and shows how it is used to implement Vec, and then how Vec is end up being used, it will be very hard to tell.

gnzlbg (Nov 29 2019 at 15:40, on Zulip):

I'm skeptic that it will be as useful as @John Ericson says, but I'm also skeptic about this being completely useless, as @Simon Sapin seems to believe. I'll probably only be able to tell when I see both options next to each other.

gnzlbg (Nov 29 2019 at 15:40, on Zulip):

Vec::try_reserve is a solution that's a bit "in-between", and maybe it is the best solution there is. No idea yet.

John Ericson (Nov 30 2019 at 04:02, on Zulip):

@Tim Diekmann I am working on my allog-wg, why does Box use unchecked stuff?

Tim Diekmann (Nov 30 2019 at 10:37, on Zulip):

That's a good catch. I forgot to remove it when I removed AbortAlloc, thanks

Tim Diekmann (Nov 30 2019 at 10:56, on Zulip):

You basically reverted my AbortAlloc :D

John Ericson (Nov 30 2019 at 14:24, on Zulip):

@Tim Diekmann I tried to find AbortAlloc but didn't, thanks

Tim Diekmann (Nov 30 2019 at 14:25, on Zulip):

I removed it at 0.7.0. You can find it in the docs for 0.6.0

John Ericson (Nov 30 2019 at 14:25, on Zulip):

@Tim Diekmann OK I read your explanation, thanks, and I think I get it now

John Ericson (Nov 30 2019 at 14:26, on Zulip):

So this goes back to what @gnzlbg said about it might be best to change the meaning of try_reserve

Tim Diekmann (Nov 30 2019 at 14:26, on Zulip):

@gnzlbg and I thought about another solution yesterday, you can see the discussion above :)

John Ericson (Nov 30 2019 at 14:27, on Zulip):

FallibleVec?

Tim Diekmann (Nov 30 2019 at 14:27, on Zulip):

I think the current meaning of try_reserve is intuitive, don't you?

John Ericson (Nov 30 2019 at 14:27, on Zulip):

Well let's just talk about FallibleVec for a second

Tim Diekmann (Nov 30 2019 at 14:28, on Zulip):

- FallibleVec, but this introduces a lot more structures
- Infallible as another alloc trait is another option

Tim Diekmann (Nov 30 2019 at 14:28, on Zulip):

sure!

John Ericson (Nov 30 2019 at 14:28, on Zulip):

So we have this split ecosytem that's part "I don't care about allocation failure" and "I really care about allocation failure"

Tim Diekmann (Nov 30 2019 at 14:28, on Zulip):

Yeah, true

John Ericson (Nov 30 2019 at 14:28, on Zulip):

I want there to be libraries that support both

Tim Diekmann (Nov 30 2019 at 14:29, on Zulip):

Me too. The best case would be an API which supports both

John Ericson (Nov 30 2019 at 14:29, on Zulip):

My best idea for that is the with the foo = try_foo.infallible_unwrap() pattern

John Ericson (Nov 30 2019 at 14:29, on Zulip):

Is FallibleVec supposed to be a whole different struct?

John Ericson (Nov 30 2019 at 14:30, on Zulip):

Maybe a more compatible with the above alternative would be an extension trait that you don't get in the prelude, so regular users aren't confused?

John Ericson (Nov 30 2019 at 14:30, on Zulip):

Also I think a type Panic = ! would make things a lot more intuitive?

Tim Diekmann (Nov 30 2019 at 14:30, on Zulip):

Maybe a more compatible with the above alternative would be an extension trait that you don't get in the prelude, so regular users aren't confused?

I thought about that, too

Tim Diekmann (Nov 30 2019 at 14:31, on Zulip):

Also I think a type Panic = ! would make things a lot more intuitive?

Yes, indeed. That's why I really liked the AllocRef<Error = !> approach

John Ericson (Nov 30 2019 at 14:31, on Zulip):

If you see fn try_reserve(&mut self) -> Result<(), Panic>, <- after subsitutition, I think it makes a bit more sense what is going on.

John Ericson (Nov 30 2019 at 14:35, on Zulip):

@Tim Diekmann so back to the test failures, is this just a matter of changing the tests?

Tim Diekmann (Nov 30 2019 at 14:35, on Zulip):

In the current state (0.7.0), you can just call reserve and it will abort on OOM. However, when calling try_reserve it will return Result<(), Err>. If your allocator never fails, it still returns Return<(), !>, which is pretty intuitive. The user, who don't care about fallible collections can still use reserve. Do I get you right, that you want to introduce all the try_ methods in an extension traits excluded from the prelude?

Tim Diekmann (Nov 30 2019 at 14:36, on Zulip):

You can fix this when exchanging AbortAlloc<Global> (forgot your name :) ) with Global. This requires a bit of dancing for the String tests, but it's possible. Let me search the commit, where I fixed this test - I also had those failures before.

John Ericson (Nov 30 2019 at 14:38, on Zulip):

@Tim Diekmann also in general, sorry for making a bug fuss and not finding the commits that showed you had tried all of the stuff in that PR.

John Ericson (Nov 30 2019 at 14:39, on Zulip):

I've wasted both our time :grimacing:

John Ericson (Nov 30 2019 at 14:39, on Zulip):

I found the tests commit now

Tim Diekmann (Nov 30 2019 at 14:39, on Zulip):

for Vec: https://github.com/TimDiekmann/alloc-wg/pull/10/commits/dfa9939e4bd6007fcd9d5a89537d3081ef1634a7
for String: https://github.com/TimDiekmann/alloc-wg/pull/10/commits/06fff0335cc00abf195659056cb52286b43b43d5

Tim Diekmann (Nov 30 2019 at 14:39, on Zulip):

Me too :yum:

Tim Diekmann (Nov 30 2019 at 14:40, on Zulip):

I think you better "waste" my time for one hour, than don't participate at all :)
Now you know about my design decisions, so don't worry :+1:

Tim Diekmann (Nov 30 2019 at 14:41, on Zulip):

Don't know if the second commit is required now, but the first definitely fixed your test case

Tim Diekmann (Nov 30 2019 at 14:43, on Zulip):

But back to the design: I don't think we should stick with AbortAlloc/PanicAdapter at all. Do you want to try out, how the extension trait works out?
Personally, I'd love to have more traits on collections anyway, but this is another story.

Tim Diekmann (Nov 30 2019 at 14:44, on Zulip):

Like generic traits for slicing, random access, len etc.

Tim Diekmann (Nov 30 2019 at 14:44, on Zulip):

But this wasn't done yet due to the lack of GAT

John Ericson (Nov 30 2019 at 15:06, on Zulip):

GAT?

Tim Diekmann (Nov 30 2019 at 15:07, on Zulip):

Generic associated type

John Ericson (Nov 30 2019 at 15:07, on Zulip):

ah ok

John Ericson (Nov 30 2019 at 15:07, on Zulip):

yeah it is ugly without it

John Ericson (Nov 30 2019 at 15:07, on Zulip):

So if you like type Panic = !, what is the remaing issue with the adapter?

John Ericson (Nov 30 2019 at 15:08, on Zulip):

Or really, I am much more fond of the Error = ! bounds than the adapter itself

John Ericson (Nov 30 2019 at 15:08, on Zulip):

I combined my work with the reverts or reverts, and the tests pass now

John Ericson (Nov 30 2019 at 15:08, on Zulip):

fwiw

Tim Diekmann (Nov 30 2019 at 15:09, on Zulip):

Calling try_reserve with AbortAlloc has an unintuitive behavior: It aborts on OOM. When AbortAlloc is the default for Vec, try_reserve is pretty useless for non-custom allocators

Tim Diekmann (Nov 30 2019 at 15:11, on Zulip):

The expected behavior would be, that it aborts on reserve and returns a result on try_reserve

Tim Diekmann (Nov 30 2019 at 15:11, on Zulip):

But without an additional trait or more methods in ReallocRef this isn't possible

John Ericson (Nov 30 2019 at 15:12, on Zulip):

@Tim Diekmann Are you just worried about try_reserve, or all such try_* methods?

Tim Diekmann (Nov 30 2019 at 15:12, on Zulip):

All try_* methods

John Ericson (Nov 30 2019 at 15:13, on Zulip):

So we could do some associated type monstrosity to fix this

Tim Diekmann (Nov 30 2019 at 15:13, on Zulip):

But try_reserve is the most intuitive example. What's the point on having Vec::<T, AbortAlloc<Global>>::try_reserve if it only checks for capacity overflow?

Tim Diekmann (Nov 30 2019 at 15:14, on Zulip):

so, this would be

let mut v = vec![];
v.try_reserve(10)?; // may abort
John Ericson (Nov 30 2019 at 15:15, on Zulip):

Yes, the point is just so reserve can be written in terms of try_reserve

John Ericson (Nov 30 2019 at 15:15, on Zulip):

BTW, how does capacity overlow happen without an alloc failure? Shouldn't it be underflow?

Tim Diekmann (Nov 30 2019 at 15:15, on Zulip):

You can still do this, but you get the desired behavior when not using AbortAlloc and dropping the Error = ! bound

Tim Diekmann (Nov 30 2019 at 15:17, on Zulip):

capacity overflow may only occure on 16bit and 32 bit platforms, as only isize capacity is allowed there. If you try to reserve isize::MAX / 4 slots with size_of::<T>() == 8 it will overflow

John Ericson (Nov 30 2019 at 15:18, on Zulip):

Wouldn't that also be an allocation failure?

John Ericson (Nov 30 2019 at 15:18, on Zulip):

err nevermind

John Ericson (Nov 30 2019 at 15:19, on Zulip):

@Tim Diekmann So honestly if what is in alloc-wg landed soon, and the try_* methods were unstable until GAT, I would be perfectly happy

John Ericson (Nov 30 2019 at 15:19, on Zulip):

what do you think about that?

Tim Diekmann (Nov 30 2019 at 15:24, on Zulip):

Wouldn't that also be an allocation failure?

Yes, but currently CapacityOverflow simply panics in RawVec::reserve. In RawVec::try_reserve it returns Err(CapacityAllocError::CapacityOverflow).
Vec::try_reserve also returns Result<(), CapacityAllocError>, but only the CapacityOverflow variant.

Tim Diekmann (Nov 30 2019 at 15:25, on Zulip):

Even if the alloc-wg stuff will be merged upstream, it will take at least a year I guess until anything is stabilized. GATs are not required for the try_* methods, only for other trait methods on collections, but this is another story and probably out of scope for this WG (currently).

John Ericson (Nov 30 2019 at 15:26, on Zulip):

It might be for like clone_in and other multi-allocator ones?

Tim Diekmann (Nov 30 2019 at 15:28, on Zulip):

No, generic trait methods for more collections. CloneIn is possible without GATs, too. However, even if a trait would need GATs I'd also be fine waiting for it. We shouldn't make the API more complicated than necessary, only because an approved language feature isn't completed yet.

John Ericson (Nov 30 2019 at 15:29, on Zulip):

sorry try_clone_in if we wanted to somehow gate that too

Tim Diekmann (Nov 30 2019 at 15:30, on Zulip):

try_clone_in :slight_smile:

John Ericson (Nov 30 2019 at 15:31, on Zulip):

Oh right, that workaround

John Ericson (Nov 30 2019 at 15:31, on Zulip):

And an additional trait could also use it

Tim Diekmann (Nov 30 2019 at 15:31, on Zulip):

master doc: https://timdiekmann.github.io/alloc-wg/alloc_wg/clone/trait.CloneIn.html#tymethod.try_clone_in

Tim Diekmann (Nov 30 2019 at 15:33, on Zulip):

But I'm unsure about the error type there. The error type may should be associated with the CloneIn trait itself, so we can return CollectionAllocErr to retrieve the layout

Tim Diekmann (Nov 30 2019 at 15:33, on Zulip):

This is dropped here

John Ericson (Nov 30 2019 at 15:34, on Zulip):

Ah that

Tim Diekmann (Nov 30 2019 at 15:34, on Zulip):

@John Ericson To finish this topic, do you agree that AbortAlloc makes no sense like this?

John Ericson (Nov 30 2019 at 15:35, on Zulip):

I think it makes sense if you are very used to !, but I'm fine not doing it unless it can be made less surprising for everything that isn't.

John Ericson (Nov 30 2019 at 15:36, on Zulip):

The big battle here is just getting try_ methods on the collections at all

Tim Diekmann (Nov 30 2019 at 15:36, on Zulip):

Yeah, it's a nice thing regarding !, but incompatible with try_*.

John Ericson (Nov 30 2019 at 15:37, on Zulip):

So without the Error = ! bounds, I suspect people will push to drop the associated error type altogether?

Tim Diekmann (Nov 30 2019 at 15:37, on Zulip):

However, Error=! as bound on reserve and other non-try_ methods is just a way to force the user to use try instead.

John Ericson (Nov 30 2019 at 15:38, on Zulip):

Yes, it is nice to pass in a collection to a library that is allocator polymorphic, and force it to do the right thing

Tim Diekmann (Nov 30 2019 at 15:38, on Zulip):

So without the Error = ! bounds, I suspect people will push to drop the associated error type altogether?

Yes, another thing I totally forgot: You would have need to wrap nearly all allocators in AbortAlloc to just ignore the associated error. Just as you mentioned, the ecosystem is split there.

Tim Diekmann (Nov 30 2019 at 15:39, on Zulip):

It's a nice thing, if you want to never panic/abort. But that's only the unusual case.

Tim Diekmann (Nov 30 2019 at 15:39, on Zulip):

Normal programs will never recover from OOM anyway.

John Ericson (Nov 30 2019 at 15:40, on Zulip):

We can still support both ecosystem with the approach on master today

Tim Diekmann (Nov 30 2019 at 15:40, on Zulip):

And the capacity overflow panic is a bug in the program

John Ericson (Nov 30 2019 at 15:40, on Zulip):

Just you are really reliant on checking for master by hand

John Ericson (Nov 30 2019 at 15:40, on Zulip):

But I'll admit that any library should be able to do the foo try_foo duplication with the panic rather than Error = ! bound

John Ericson (Nov 30 2019 at 15:41, on Zulip):

so the same libraries can support both still

John Ericson (Nov 30 2019 at 15:41, on Zulip):

Yeah I am much more sympathetic to capacity overflow always being a regular overflow panic

John Ericson (Nov 30 2019 at 15:42, on Zulip):

and well, if you don't want to panic run a model checker or something

John Ericson (Nov 30 2019 at 15:42, on Zulip):

as you shouldn't sanitize uncontrolled values by catching those errors, since as you say this program really is broken whereas normal allocation failure can be not the program's fault

Tim Diekmann (Nov 30 2019 at 15:43, on Zulip):

Another Idea: I think we are able to support both ecosystems this way: ...

Tim Diekmann (Nov 30 2019 at 15:44, on Zulip):

- Introduce a marker trait Abort
- for reserve and allocating non-try_* methods add this as bound
- Implement Abort on Global and System

Tim Diekmann (Nov 30 2019 at 15:44, on Zulip):

That's basically the inversion of AbortAlloc, right?

Tim Diekmann (Nov 30 2019 at 15:45, on Zulip):

And if you want to be forced to check OOM, you just have to wrap your allocator but don't implement Abort

John Ericson (Nov 30 2019 at 15:45, on Zulip):

:)

Tim Diekmann (Nov 30 2019 at 15:45, on Zulip):

That also removes the very missleading term of "infallible"

Tim Diekmann (Nov 30 2019 at 15:45, on Zulip):

Global isn't infallible, we want to ignore the error

Tim Diekmann (Nov 30 2019 at 15:46, on Zulip):

Abort allows the allocator to fail

John Ericson (Nov 30 2019 at 15:46, on Zulip):

the one thing is it's the collection doing the panicking, not the allocator, right?

John Ericson (Nov 30 2019 at 15:46, on Zulip):

I think you want Abort and AbortAlloc

Tim Diekmann (Nov 30 2019 at 15:47, on Zulip):

Yes, I think so

Hmm, let's see. let me write a few lines of code...

John Ericson (Nov 30 2019 at 15:47, on Zulip):

if you have an A: Abort Then it does a cast to AbortAlloc<A> and calls the method with that?

Tim Diekmann (Nov 30 2019 at 15:49, on Zulip):
trait Abort {}

impl Abort for Global {}

impl<T, A: DeallocRef> Vec<T, A> {
    pub fn reserve(&mut self, additional: usize)
    where
        A: ReallocRef + Abort,
    {
        match self.try_reserve(used_capacity, needed_extra_capacity) {
            Ok(vec) => vec,
            Err(CollectionAllocErr::CapacityOverflow) => capacity_overflow(),
            Err(CollectionAllocErr::AllocError { layout, .. }) => handle_alloc_error(layout.into()),
        }
    }

    pub fn try_reserve(&mut self, additional: usize) -> Result<(), CollectionAllocErr<A>>
    where
        A: ReallocRef,
    {
        self.buf.try_reserve(self.len, additional)
    }
}
Tim Diekmann (Nov 30 2019 at 15:51, on Zulip):
// Don't implement `Abort` on `SaveAlloc`
struct SaveAlloc<A>(A);

// impl `AllocRef` etc. for `SaveAlloc`
John Ericson (Nov 30 2019 at 15:51, on Zulip):

oh nice!

John Ericson (Nov 30 2019 at 15:51, on Zulip):

Yeah that's good, you can make a method that also requires Abort which calls handle_alloc_error

Tim Diekmann (Nov 30 2019 at 15:52, on Zulip):

Yeah, true! nice!

John Ericson (Nov 30 2019 at 15:52, on Zulip):

the A is sort of phantom as it takes an Result<T, A::Error>

John Ericson (Nov 30 2019 at 15:52, on Zulip):

and another for Result<T, CollectionsError<A::Error>>

Tim Diekmann (Nov 30 2019 at 15:53, on Zulip):

Wow, I'm very excited, I think this is really a very good solution :)

John Ericson (Nov 30 2019 at 15:54, on Zulip):

me too! I was worried it would be endless type machinary, but this isn't even that bad!

Tim Diekmann (Nov 30 2019 at 15:54, on Zulip):

It's just a single marker trait. We already got plenty marker traits anyway

John Ericson (Nov 30 2019 at 15:55, on Zulip):

yeah that's not a controversial thing by now

John Ericson (Nov 30 2019 at 15:55, on Zulip):

If you want to delegate to me to write it up, then you can get your time back with me rehashing the old AbortAlloc :)

Tim Diekmann (Nov 30 2019 at 15:56, on Zulip):

I have to go now anyway. If you want to do this, go ahead!

Tim Diekmann (Nov 30 2019 at 15:56, on Zulip):

I'll close your current PR then?

John Ericson (Nov 30 2019 at 15:57, on Zulip):

Yeah

Tim Diekmann (Nov 30 2019 at 15:58, on Zulip):

Hey:

impl<T> Abort for T where T: AllocRef<Error = !> {}
John Ericson (Nov 30 2019 at 15:58, on Zulip):

Yes can do that too, though will still need the separate Global and System ones

Gurwinder Singh (Nov 30 2019 at 15:59, on Zulip):

:+1:

Tim Diekmann (Nov 30 2019 at 16:00, on Zulip):

Yes can do that too, though will still need the separate Global and System ones

Sure, but at least you don't have to implement Abort on any new allocators :)

Tim Diekmann (Nov 30 2019 at 16:04, on Zulip):

Isn't this a good point to keep the associated error type?

John Ericson (Nov 30 2019 at 16:06, on Zulip):

Yes it is!

John Ericson (Nov 30 2019 at 16:07, on Zulip):

@Tim Diekmann OK so first issue is I think we do need AbortAlloc with this, because in general don't have the layout for handle_alloc_error

John Ericson (Nov 30 2019 at 16:07, on Zulip):

this is partially why the boxed ones still use unwrap_unchecked by mistake

John Ericson (Nov 30 2019 at 16:07, on Zulip):

it's a lot less mechanical to fix

John Ericson (Nov 30 2019 at 16:07, on Zulip):

but AbortAlloc can be private

Tim Diekmann (Nov 30 2019 at 16:25, on Zulip):

Yes, this would replace the AbortAlloc thing entirely.

I don't have more time today to head more into this, I'll look more into this tomorrow :)
Just go ahead :)

gnzlbg (Nov 30 2019 at 17:34, on Zulip):

FWIW the FallibleVec would be what Vec is today

gnzlbg (Nov 30 2019 at 17:34, on Zulip):

and Vec would just become a "thin" wrapper over FallibleVec that .unwraps() on every method

gnzlbg (Nov 30 2019 at 17:36, on Zulip):

There wouldn't be any _try methods

gnzlbg (Nov 30 2019 at 17:36, on Zulip):

You would just pick Vec or FallibleVec depending on whether you want an infallible or fallible API

Tim Diekmann (Nov 30 2019 at 17:45, on Zulip):

Yes, but it would introduce many many new structs

John Ericson (Nov 30 2019 at 18:10, on Zulip):

You would just pick Vec or FallibleVec depending on whether you want an infallible or fallible API

@gnzlbg beyond the many new structs, it would be much harder to reuse the implementation

John Ericson (Nov 30 2019 at 18:12, on Zulip):

https://github.com/TimDiekmann/alloc-wg/pull/16 is now up with @Tim Diekmann's Abort trait idea. I had to put in AllocAbort as an implementation detail, but a better method might be changing the error types to be something ~~ (AllocError, NonZeroLayout)

John Ericson (Nov 30 2019 at 18:13, on Zulip):

I say we try to get this one merged with the private AllocAbort, and then make another issue for the error handling, as the temporary AllocAbort is good to make the issue clear.

gnzlbg (Dec 02 2019 at 09:17, on Zulip):

@John Ericson by many new structs you mean, ~5 structs ? (Vec, List, HashMap, BTreeSet, Deque) ?

gnzlbg (Dec 02 2019 at 09:17, on Zulip):

Also, why do you think it would be harder to reuse the implementation ?

gnzlbg (Dec 02 2019 at 09:17, on Zulip):

You can reuse Vec like you do today, and for FallibleVec it would be just the same.

Tim Diekmann (Dec 02 2019 at 14:37, on Zulip):

At least those 5 structs + strings + any other structs that will be associated with an allocator.
Additionally, every api addition has to be added to both structs, which is very error prone. Also, it's a huge documentation bloat. I don't see the advantages of FallibleVec in comparison to the Abort trait or the current solution.

Tim Diekmann (Dec 02 2019 at 14:38, on Zulip):

Tracking Issue for structs which needs an allocator
This would introduce at least 15(!) new structs for a rarely used API.

gnzlbg (Dec 02 2019 at 14:43, on Zulip):

Yeah those 15 types would be pretty much it

gnzlbg (Dec 02 2019 at 14:44, on Zulip):

Many API additions already need to be added to multiple types, e.g., to Vec and RawVec, particularly when they interact with allocators

gnzlbg (Dec 02 2019 at 14:44, on Zulip):

notice that this duplication of the API is only required for functions that could allocate

gnzlbg (Dec 02 2019 at 14:44, on Zulip):

so maybe there is a better way to land it than duplicating all structs

gnzlbg (Dec 02 2019 at 14:45, on Zulip):

I doubt that though, because while it is possible to write an infallible API on top of a fallible one, e.g., using a trait, you can't write a fallible API on top of an infallible one

gnzlbg (Dec 02 2019 at 14:46, on Zulip):

also, with a stable allocator interface, if somebody wants a fallible vector, they can just use one from crates.io

gnzlbg (Dec 02 2019 at 14:47, on Zulip):

doing that this way literally requires duplicating most of the code, but it can be done

John Ericson (Dec 02 2019 at 15:00, on Zulip):

@gnzlbg I'm confused. There's now way to write arbitrary vec-polymorphic code without higher kinded types, so having multiple vecs seems like a disaster? What problems do you have with https://github.com/TimDiekmann/alloc-wg/pull/16 ?

John Ericson (Dec 02 2019 at 15:03, on Zulip):

Also confused about the crates.io bit. I agree it would be good if more collections came from crates.io—std can even have private deps and re-export. But this is totally separate from allocator stuff? I.e. we should "pull a hash brown" for everything else anyways?

gnzlbg (Dec 02 2019 at 15:18, on Zulip):

What problems do you have with https://github.com/TimDiekmann/alloc-wg/pull/16 ?

If I'm writing a library that should propagate all alloaction errors to users, and my library uses an API like Vec::extend, how do I propagate the error ?

gnzlbg (Dec 02 2019 at 15:21, on Zulip):

There's now way to write arbitrary vec-polymorphic code without higher kinded types, so having multiple vecs seems like a disaster?

Why would you need that ?

gnzlbg (Dec 02 2019 at 15:21, on Zulip):

If you have an API that should accept / return multiple vector types, like Vec, SmallVec, or ArrayVec, then you can write a VecLike trait, and implement it for those types.

gnzlbg (Dec 02 2019 at 15:22, on Zulip):

I don't see how another kind of Vec makes this problem worse.

John Ericson (Dec 02 2019 at 15:33, on Zulip):

If I'm writing a library that should propagate all alloaction errors to users, and my library uses an API like Vec::extend, how do I propagate the allocation error to a user such that it can recover ?

You write a mylib::try_method with Vec::try_extend, and then in the same manner make your mylib::method in terms of mylib::try_method.

John Ericson (Dec 02 2019 at 15:34, on Zulip):

I don't see how another kind of Vec makes this problem worse.

I don't see how another kind of Vec makes this problem worse. It doesn't make the problem worse, but it's already bad. There's no good way to write VecLike today, and generic associated types only make that slightly better.

Tim Diekmann (Dec 02 2019 at 17:05, on Zulip):

I reviewed your PR @John Ericson. Thanks again for implementing it :)

gnzlbg (Dec 03 2019 at 15:41, on Zulip):

@John Ericson

You write a mylib::try_method with Vec::try_extend

There is no try_extend method.

gnzlbg (Dec 03 2019 at 15:41, on Zulip):

And that approach would require duplicating most of the API of all collections

gnzlbg (Dec 03 2019 at 15:42, on Zulip):

I don't see how that is better than having two types providing the different APIs, and without the try_ suffixes

gnzlbg (Dec 03 2019 at 15:43, on Zulip):

It doesn't make the problem worse, but it's already bad. There's no good way to write VecLike today, and generic associated types only make that slightly better.

I see that as an orthogonal issue. The argument that we should not do this because it would make something that's already impossible to do not harder isn't very strong.

Tim Diekmann (Dec 09 2019 at 11:56, on Zulip):

I actually tried out a FallibleVec. Besides my previous concerns, I also went into a no-go: You cannot trivially switch between a fallible and an aborting allocation, you have to know beforehand, if all OOMs aborts or return a Result.

Tim Diekmann (Dec 09 2019 at 11:59, on Zulip):

With the Abort marker trait, you can chose, if you allow aborting: you have to implement Abort for your allocator. If you don't implement it, the collection cannot abort. But if you do, you can chose, if your current allocation is allowed to abort or not.

Last update: Dec 12 2019 at 01:15UTC