Stream: general

Topic: __int128


gnzlbg (Aug 10 2019 at 15:42, on Zulip):

@nagisa maybe this fixed the issue you mention: https://bugs.llvm.org/show_bug.cgi?id=31362 it specifically covers:

__attribute__((ms_abi, noinline)) __int128 passthrough_a(__int128 a) {
    return a;
}
gnzlbg (Aug 10 2019 at 15:43, on Zulip):

but that was fixed in clang 6.0

gnzlbg (Aug 10 2019 at 15:43, on Zulip):

if you have seen a mismatching ABI since, then it is probably worth filling a bug for that

nagisa (Aug 10 2019 at 17:45, on Zulip):

Well the resulting codegen may match gcc now, but both are still incorrect according to the documentation so /me shrugs.

nagisa (Aug 10 2019 at 17:47, on Zulip):

I guess your point still stands that if we only intend to ever interoperate with gcc and clang, then we can match their behaviour

rkruppe (Aug 10 2019 at 17:52, on Zulip):

But we do have explicitly MSVC-compatible targets. And I don't want to end up with an extern "C" ABI break on those targets if MSVC adds __int128 support in a way incompatible with gcc/llvm.

rkruppe (Aug 10 2019 at 17:53, on Zulip):

There's still other targets without any __int128 support, so we need a way for the FFI-safety of i128 to be target-dependent anyway. The same mechanism should be used to make it not-FFI-safe on the *-msvc targets.

nagisa (Aug 10 2019 at 18:22, on Zulip):

The issue with making ffi-safety conditional is that people only ever test their code on x86_64 :slight_smile:

rkruppe (Aug 10 2019 at 18:31, on Zulip):

I'm not saying it's a great solution, but it's the only possibility if we want to make it ffi-safe at all¯\_(ツ)_/¯

Lokathor (Aug 10 2019 at 22:51, on Zulip):

I build test on x86/x86_64/wasm32, and also build on arm6t, arm7, and aarch64

Lokathor (Aug 10 2019 at 22:51, on Zulip):

but I'm not sure that any of that assures that an ABI incompatibility is caught

Lokathor (Aug 10 2019 at 22:52, on Zulip):

Retep once described ABI mismatch as "a link error, if you're lucky"

nagisa (Aug 10 2019 at 23:02, on Zulip):

WindowsBunny forgot to add a few “very” in between of “you’re” and "lucky”.

nagisa (Aug 10 2019 at 23:03, on Zulip):

Link error will only happen if you have a function that is defined using calling convention A, and try to refer to it with a declaration that uses calling convention B. And that applies to very limited list of calling convention pairs.

gnzlbg (Aug 11 2019 at 11:23, on Zulip):

And I don't want to end up with an extern "C" ABI break on those targets if MSVC adds __int128 support in a way incompatible with gcc/llvm.

@rkruppe why not? I think this is fine.

gnzlbg (Aug 11 2019 at 11:24, on Zulip):

No existing code would break, because there is no existing code interfacing with MSVC code, because that can't be done.

gnzlbg (Aug 11 2019 at 11:24, on Zulip):

Once gcc & clang update to match that ABI, we can do the same, offering some kind of backward compat flag, if that turns out to be necessary.

gnzlbg (Aug 11 2019 at 11:25, on Zulip):

Chances are, MSVC will never support __int128, and if they do, they could do so in a way that is compatible with all other code on their platform that's already using __int128.

gnzlbg (Aug 11 2019 at 11:26, on Zulip):

If they decide not to do that, then well, that's actually no difference at all than when they decide to break their ABI on every new MSVC release anyways.

rkruppe (Aug 11 2019 at 11:26, on Zulip):

It's not true no existing code would break, e.g. every Rust project using cdylib with a C ABI interface (as we recommend for plugin-like usage) would be affected by the ABI break

gnzlbg (Aug 11 2019 at 11:27, on Zulip):

How ? That code is not interfacing with MSVC compiled code.

gnzlbg (Aug 11 2019 at 11:27, on Zulip):

That code only breaks when we upgrade our ABI

gnzlbg (Aug 11 2019 at 11:27, on Zulip):

and we can offer a backward compatibility flag to choose an ABI, like all other compilers do on ABI changes

rkruppe (Aug 11 2019 at 11:28, on Zulip):

The ABI break we're talking about is us changing our implementation of extern "C" on msvc compatible targets to be actually msvc compatible

gnzlbg (Aug 11 2019 at 11:28, on Zulip):

-C abi-compat=1.X.Y

gnzlbg (Aug 11 2019 at 11:28, on Zulip):

Sure. If MSVC changes their ABI, we have to do a backward breaking change

rkruppe (Aug 11 2019 at 11:28, on Zulip):

This is always a shitty solution and I would MUCH rather avoid the ABI break to begin with

gnzlbg (Aug 11 2019 at 11:28, on Zulip):

currently, we do this things all the time

gnzlbg (Aug 11 2019 at 11:28, on Zulip):

without a flag

rkruppe (Aug 11 2019 at 11:28, on Zulip):

C ABI breaks are very rare and always cause a lot of pain

gnzlbg (Aug 11 2019 at 11:28, on Zulip):

because we can't support different versions of a platform

gnzlbg (Aug 11 2019 at 11:28, on Zulip):

clang and gcc also do this on pretty much every release

rkruppe (Aug 11 2019 at 11:29, on Zulip):

Nonsense

rkruppe (Aug 11 2019 at 11:29, on Zulip):

C++ Abis change, the rust abi changes, but C ABI changes are exceedingly rare

gnzlbg (Aug 11 2019 at 11:30, on Zulip):

https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-fclang-abi-compat

gnzlbg (Aug 11 2019 at 11:30, on Zulip):

they do happen for C as well

rkruppe (Aug 11 2019 at 11:30, on Zulip):

Yes but extremely rarely and usually only for obscure corner cases or bug fixes

gnzlbg (Aug 11 2019 at 11:30, on Zulip):

sure

gnzlbg (Aug 11 2019 at 11:31, on Zulip):

the MSVC ABI has a bug, because compilers for the target use a __int128 type, yet the ABI spec does not document it

gnzlbg (Aug 11 2019 at 11:31, on Zulip):

that would be a bug fix

gnzlbg (Aug 11 2019 at 11:32, on Zulip):

ABI specs aren't a document that compilers implement, ABI specs are a document that attempt to document what some compiler on a platform does, so that other compilers can work with it

rkruppe (Aug 11 2019 at 11:32, on Zulip):

C ABI breaks are done when you ended up in a shitty situation by accident, not something to willingly walk into as "fine"

gnzlbg (Aug 11 2019 at 11:32, on Zulip):

clang and gcc on the target do the sane thing

gnzlbg (Aug 11 2019 at 11:33, on Zulip):

msvc doesn't support the use case at all, doing what clang and gcc do is fine

gnzlbg (Aug 11 2019 at 11:33, on Zulip):

that's the only ABI that exist for __int128 on the target

gnzlbg (Aug 11 2019 at 11:33, on Zulip):

Microsoft documenting and implementing a different ABI for it would be doing the shitty thing

gnzlbg (Aug 11 2019 at 11:33, on Zulip):

is their target, if they want to break it, then we can't do anything about that

gnzlbg (Aug 11 2019 at 11:33, on Zulip):

clang and gcc would need a breaking change there as well, such is life

rkruppe (Aug 11 2019 at 11:34, on Zulip):

That is a good point I guess

gnzlbg (Aug 11 2019 at 11:34, on Zulip):

its an unfortunate situation to be in, we should open a microsoft bug, and instead of asking for msvc to implement __int128, just ask them to document the ABI of the type

gnzlbg (Aug 11 2019 at 11:35, on Zulip):

if the spec doesn't mention it, the spec has a bug, it doesn't matter that msvc doesn't support it

gnzlbg (Aug 11 2019 at 11:35, on Zulip):

other compilers for the target do, and they should be able to agree on the ABI

gnzlbg (Aug 11 2019 at 11:35, on Zulip):

all other sane ABI documents for 64-bit targets document this as well

rkruppe (Aug 11 2019 at 11:36, on Zulip):

But backtracking a little, why do we want i128 ffi safety on the msvc targets anyway when it can't be used with msvc code? Why not treat the windows-msvc triples as not having __int128 (which is mostly true)? Does any user ask for compatibility with gcc/clang ms_abi compatibility for this type?

gnzlbg (Aug 11 2019 at 11:38, on Zulip):

But backtracking a little, why do we want i128 ffi safety on the msvc targets anyway when it can't be used with msvc code?

To use it with clang or gcc.

gnzlbg (Aug 11 2019 at 11:39, on Zulip):

you can use clang via clang-cl.exe to compile for msvc targets, and you can use __int128 there

gnzlbg (Aug 11 2019 at 11:39, on Zulip):

Also, to use it with other Rust code that uses i128, like plugins, dlls, etc.

rkruppe (Aug 11 2019 at 11:39, on Zulip):

I know that is possible. But does anyone ask for that?

rkruppe (Aug 11 2019 at 11:40, on Zulip):

For that target specifically

gnzlbg (Aug 11 2019 at 11:40, on Zulip):

Most bugs are about "why can't i use i128 on FFI", we haven't surveyed what people wanted to do with it

gnzlbg (Aug 11 2019 at 11:41, on Zulip):

There are obviously no libraries compiled with msvc that use i128

gnzlbg (Aug 11 2019 at 11:41, on Zulip):

but it is trivial to use gcc or clang from visual studio nowadays, and a lot of people prefer that, so I wouldn't assume anything without doing a survey of usecases

rkruppe (Aug 11 2019 at 11:42, on Zulip):

Right but there will always be many targets where i128 can't be ffi safe because there's no C ABI for it in any compiler for the platform, so "I want to use i128 in cross platform rust-rust C-ABI communication" can't be supported anyway

rkruppe (Aug 11 2019 at 11:42, on Zulip):

i128 in ffi is unfortunately going to be a platform specific affair

gnzlbg (Aug 11 2019 at 11:42, on Zulip):

For me, personally, I think that "FFI-safety" means we provide the type an ABI that can be used to interface with other C code compiled for the platform

gnzlbg (Aug 11 2019 at 11:43, on Zulip):

that doesn't mean that all compilers for a platform support everything, or that they support the same thing in the same way

gnzlbg (Aug 11 2019 at 11:43, on Zulip):

it just means that it is possible to write code that relies on the ABI of the type

gnzlbg (Aug 11 2019 at 11:44, on Zulip):

if you are writing msvc code that needs to interface with rust i128, you might be able to match its ABI by using a struct __int128 { int64_ hi, lo; }; to interface with it.

rkruppe (Aug 11 2019 at 11:44, on Zulip):

If multiple compilers implement the corresponding C type differently, we have a problem in that we need to pick which one we're compatible with. Which may mean refusing to choose, as we do so far re: the interaction of repr(align) and repr(packed)

gnzlbg (Aug 11 2019 at 11:44, on Zulip):

yep, there we have multiple compilers doing it all in different ways

gnzlbg (Aug 11 2019 at 11:45, on Zulip):

IIRC msvc supports it, gcc does something different, and clang does what gcc does

gnzlbg (Aug 11 2019 at 11:45, on Zulip):

that's imo a gcc and clang bug that should be fixed

gnzlbg (Aug 11 2019 at 11:45, on Zulip):

i don't know if we actually opened the bugs

rkruppe (Aug 11 2019 at 11:45, on Zulip):

And that's also a risk that we'll end up in that situation with __int128

gnzlbg (Aug 11 2019 at 11:46, on Zulip):

the situation is slightly different as argued above

rkruppe (Aug 11 2019 at 11:46, on Zulip):

But if we rush ahead now and claim i128 is ffi safe on that platform we won't have the option of refusing to choose

gnzlbg (Aug 11 2019 at 11:46, on Zulip):

if we want to make i128 ffi-safe on the platform, we only have one option, as opposed to 2

gnzlbg (Aug 11 2019 at 11:46, on Zulip):

for all we know, microsoft might never support it, so it might never be an issue

gnzlbg (Aug 11 2019 at 11:47, on Zulip):

we can open a microsoft documentation bug for their abi

gnzlbg (Aug 11 2019 at 11:47, on Zulip):

they support clang-cl, and clang-cl supports __in128

rkruppe (Aug 11 2019 at 11:47, on Zulip):

Once again I am talking about the possibility that msvc adds support incompatible with gcc

gnzlbg (Aug 11 2019 at 11:47, on Zulip):

sure, that's a posibility

gnzlbg (Aug 11 2019 at 11:47, on Zulip):

we can provide a x86_64-pc-windows-msvc2020 target for that posibility

gnzlbg (Aug 11 2019 at 11:48, on Zulip):

or a flag, or nothing

rkruppe (Aug 11 2019 at 11:48, on Zulip):

I'm repeating myself but that would really fucking suck!!

rkruppe (Aug 11 2019 at 11:48, on Zulip):

Let's please not risk that

gnzlbg (Aug 11 2019 at 11:48, on Zulip):

with my libc maintainer hat on, all platforms break their ABI all the time

gnzlbg (Aug 11 2019 at 11:48, on Zulip):

we just do nothing and hope for the best

gnzlbg (Aug 11 2019 at 11:49, on Zulip):

is that bad? sure, the right solution is platform ABI versioning

rkruppe (Aug 11 2019 at 11:49, on Zulip):

And it's bad enough every time, so let's not risk even more breaks

gnzlbg (Aug 11 2019 at 11:49, on Zulip):

we can not provide FFI-safety for the type for windows msvc targets

gnzlbg (Aug 11 2019 at 11:49, on Zulip):

but that has its downsides too

rkruppe (Aug 11 2019 at 11:50, on Zulip):

Once again let's check if any living person is actually affected by those hypothetical downsides

gnzlbg (Aug 11 2019 at 11:50, on Zulip):

Every person reading its docs would need to read the line "warning: this type is not FFI safe for targets x,y,z,w,"

rkruppe (Aug 11 2019 at 11:51, on Zulip):

We need that anyway, we're just talking about adding one more entry to that list

gnzlbg (Aug 11 2019 at 11:51, on Zulip):

i would provide FFI safety for all targets

rkruppe (Aug 11 2019 at 11:51, on Zulip):

i128 can't reasonably be ffi safe on most (all?) 32 bit targets

gnzlbg (Aug 11 2019 at 11:51, on Zulip):

including 32-bit targets

rkruppe (Aug 11 2019 at 11:51, on Zulip):

Even by your definition above

gnzlbg (Aug 11 2019 at 11:52, on Zulip):

no C compiler for the target supports it, so we can do whatever we want

gnzlbg (Aug 11 2019 at 11:52, on Zulip):

if a C compiler wants to allow it, we can ask them to be compatible with what we are already doing

gnzlbg (Aug 11 2019 at 11:52, on Zulip):

if we start doing it, we can just add what we are doing to the 32-bit abi specs of the platforms

rkruppe (Aug 11 2019 at 11:53, on Zulip):

FFI safety = improper_ctypes = a lint to prevent people from shooting themselves in the foot with C level interop

rkruppe (Aug 11 2019 at 11:53, on Zulip):

It's not a catch all for things whose ABI rustc guarantees!

rkruppe (Aug 11 2019 at 11:53, on Zulip):

We may want that too but it's different

gnzlbg (Aug 11 2019 at 11:53, on Zulip):

then the lint should only apply to extern "C"

gnzlbg (Aug 11 2019 at 11:53, on Zulip):

but it currently applies to all extern _s

rkruppe (Aug 11 2019 at 11:54, on Zulip):

I don't see your point

rkruppe (Aug 11 2019 at 11:54, on Zulip):

All the stable ABIs the lint applies to are C ABIs

rkruppe (Aug 11 2019 at 11:55, on Zulip):

In any case all the people who've filed bugs asking why i128 is not ffi safe were using it with extern "C"

gnzlbg (Aug 11 2019 at 11:56, on Zulip):

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=315d5078916980d2389e8c317983f4e4

rkruppe (Aug 11 2019 at 11:56, on Zulip):

lol

gnzlbg (Aug 11 2019 at 11:57, on Zulip):

I've never used an extern "ABI" for which the improper_ctypes lint has not triggered, ever

gnzlbg (Aug 11 2019 at 11:57, on Zulip):

anyways, this is another discussion

rkruppe (Aug 11 2019 at 11:57, on Zulip):

extern "Rust" blocks are practically impossible to use safely anyway

gnzlbg (Aug 11 2019 at 11:58, on Zulip):

I think that if an ABI spec does not mention a type, and its a primitive type that's reasonable to use on FFI, like a 128-bit integer, and no compilers on the target support it, we can ask the ABI spec to define an ABI for the type, and implement it

gnzlbg (Aug 11 2019 at 11:58, on Zulip):

for 32-bit linux that's as easy as sending a PR to the spec and starting a discussion about the semantics

gnzlbg (Aug 11 2019 at 11:59, on Zulip):

even if C doesn't have a type, it should be possible to write a C struct that interfaces with the type

gnzlbg (Aug 11 2019 at 11:59, on Zulip):

we have done this for things like repr(C) enums

gnzlbg (Aug 11 2019 at 11:59, on Zulip):

where C doesn't have enums

rkruppe (Aug 11 2019 at 11:59, on Zulip):

You're welcome to ask ABI authors who mostly only care about C to add non-C feature but I am not holding my breath for it to happen everywhere

rkruppe (Aug 11 2019 at 12:00, on Zulip):

even if C doesn't have a type, it should be possible to write a C struct that interfaces with the type

This is not always obviously possible depending on how you defined the i128 ABI and what the ABI for aggregates looks like

gnzlbg (Aug 11 2019 at 12:01, on Zulip):

why not? if the abi doesn't have an ABI for __int128, we can define our i128 as some C struct, allowing this

rkruppe (Aug 11 2019 at 12:02, on Zulip):

Sure you can define the ABI of i128 accordingly but you have to explicitly take care to ensure it

rkruppe (Aug 11 2019 at 12:02, on Zulip):

In any case

rkruppe (Aug 11 2019 at 12:03, on Zulip):

This idea (lobbying all ABI specs) is a huge amount of work and won't get us coverage of all platforms any time soon (if ever) so IMO the only sensible thing is to either make peace with platform dependent ffi safety or not make it ffi safe anywhere

rkruppe (Aug 11 2019 at 12:09, on Zulip):

By the way: if we go with "ffi safe on a subset of platforms" then a great way to find out if anyone needs it to be ffi safe on 64b msvc is to not include it in the list at first and wait for an issue requesting it. Then we can make a much more informed decision than right now.

gnzlbg (Aug 11 2019 at 12:20, on Zulip):

that's a good point

gnzlbg (Aug 11 2019 at 12:23, on Zulip):

The lobbying idea is an active way to make sure it works everywhere

gnzlbg (Aug 11 2019 at 12:24, on Zulip):

but at the end of the day is up to the platform to decide what to do

gnzlbg (Aug 11 2019 at 12:24, on Zulip):

we can have the warning for i128 say that the platform abi doesn't specify what to do, and document what Rust does

gnzlbg (Aug 11 2019 at 12:25, on Zulip):

on windows that would be doing what clang and gcc do

gnzlbg (Aug 11 2019 at 12:25, on Zulip):

that is, the warning is orthogonal to the implementation that we provide

gnzlbg (Aug 11 2019 at 12:26, on Zulip):

we can warn that the abi we provide for the type isn't stable, but implement the type in a sane and documented way

nagisa (Aug 11 2019 at 12:53, on Zulip):

This is RFC material.

nagisa (Aug 11 2019 at 12:53, on Zulip):

Just sayin'.

nagisa (Aug 11 2019 at 12:53, on Zulip):

The fact that you have a 100 comment back and forth shoul’ve made it pretty obvious.

Last update: Nov 20 2019 at 13:40UTC