Stream: general

Topic: Design of Duration

Nir Friedman (Jan 23 2020 at 15:35, on Zulip):

Hey, not sure if general is the best channel for this or not. I was curious about the design of Duration. Full disclosure, I'm a C++ dev that follows Rust here and there. Some conversation led me to look at the Duration struct. It seems concerning to me because by its design, it's not going to usable in very high performance applications. In C++ on the other hand, if you look at chrono's approach, you end up with a Duration type that can be a zero cost abstraction. I'm curious what the reasoning was for such a design?

oli (Jan 23 2020 at 15:36, on Zulip):

what does chrono do differently?

Nir Friedman (Jan 23 2020 at 15:37, on Zulip):

Sure, so I guess I'll give the background on both. C++ Duration is templated on basically a numeric type, which is stored, and a Ratio type. The ratio type isn't stored, but simply provides, statically, constants that tell you in what units the type is "counting" duration. It does this by providing a ratio to seconds.

Nir Friedman (Jan 23 2020 at 15:38, on Zulip):

In many high performance applications, what you typically do is simply use 64 bit integers as nanoseconds since epoch. The C++ duration type is fully compatible with this approach, and when compiled with optimizations, operations on durations will have the same codegen as if you did it using simple integers, i.e. it's a zero cost abstraction.

XAMPPRocky (Jan 23 2020 at 15:39, on Zulip):

@Nir Friedman I believe Duration's design was made long before chrono was released.

Nir Friedman (Jan 23 2020 at 15:41, on Zulip):

The rust approach is to store seconds in a 64 bit integer, and nanos in a 32 bit integer. This is simply hardcoded, which already has some disadvantages. However, in addition, it's hardcoded to something which is big (96 bytes), and also because the nanos portion needs to be kept under 1 billion, adding two durations becomes a surprisingly expensive operation. You have to add the nanos, check if the value is over a billion, if yes, then subtract a billion and add one to the seconds... etc

Nir Friedman (Jan 23 2020 at 15:43, on Zulip):

@XAMPPRocky chrono was released in 2011. Is there an easy way to see when std::time::Duration was added?

oli (Jan 23 2020 at 15:43, on Zulip):

Well... it would seem to me like there could be a generic version that Duration is just one concrete impl of

Nir Friedman (Jan 23 2020 at 15:45, on Zulip):

@oli yeah, the other thing I was going to point out is if you really badly want both massive precision and massive range, then nothing is stopping you from templating the C++ chrono on a 96 bit integer type. I'm kind of surprised that Rust avoided generics in this situation.

One thing I don't know for sure is whether Rust's type system can do what C++ does with ratio, i.e. you can have constants associated with a type? I think Rust can but I'm not certain.

simulacrum (Jan 23 2020 at 15:45, on Zulip):

You can indeed have constants associated with a type. I think prototyping a Duration that does so outside of core/std makes sense; indeed, we may want to merge it in if it works well.

simulacrum (Jan 23 2020 at 15:46, on Zulip):

(potentially, we could do so backwards compatibly, as @oli suggests)

Nir Friedman (Jan 23 2020 at 15:49, on Zulip):

@simulacrum interesting. Well, I'm not that familiar with Rust (nor honestly have the time), but if someone is interested in prototyping something like that, would be happy to talk to them, explain a bit about chrono. I think it does serve an important use case, because people in e.g. HFT simply wouldn't be able to use a Duration type like in Rust

Nir Friedman (Jan 23 2020 at 15:49, on Zulip):

Boost had a similar type (96 bit) back in the day and it was heavily avoided in HFT (high frequency trading) communities

Nir Friedman (Jan 23 2020 at 15:50, on Zulip):

I'd assume it would be similar in other fields where very low latency/high performance is necessary

XAMPPRocky (Jan 23 2020 at 15:50, on Zulip):

@Nir Friedman From researching std's duration was chrono's original duration design. Here's a link to the initial commit and the design reform.

oli (Jan 23 2020 at 15:50, on Zulip):

Boost had a similar type (96 bit) back in the day and it was heavily avoided in HFT (high frequency trading) communities

:upside_down: That sounds like a feature, not a bug

Nir Friedman (Jan 23 2020 at 15:51, on Zulip):

What was the feature? Sorry don't follow the joke

oli (Jan 23 2020 at 15:51, on Zulip):

Making life hard for HFT ppl

Nir Friedman (Jan 23 2020 at 15:53, on Zulip):

That's kinda mean, unfunny, and off topic, all at once

Nir Friedman (Jan 23 2020 at 15:54, on Zulip):

@XAMPPRocky Looking at this RFC, there's an additional pretty major limitation here, which I didn't realize, which is that Duration has to always be positive?

XAMPPRocky (Jan 23 2020 at 15:55, on Zulip):

I haven't read either two, I just found them.

Nir Friedman (Jan 23 2020 at 15:56, on Zulip):

That doesn't really make much sense, especially once you start thinking about e.g. subtracting time_point's; subtracting two points in time can easily give you a negative duration.

XAMPPRocky (Jan 23 2020 at 15:57, on Zulip):

As I understand, Duration is only for comparison between SystemTimes, it's not for general date duration.

oli (Jan 23 2020 at 15:57, on Zulip):

I think the argument is that distance can't be negative either

Nir Friedman (Jan 23 2020 at 16:02, on Zulip):

I don't really think that's much of an argument; distance is non-negative by definition because it's a magnitude. Displacement has direction, which in one dimension means signedness. You can always recover distance from displacement, but not vice versa. So you need to have a concept of the more general thing.

At any rate I don't think that such analogies add much to technical analysis, generally. But @XAMPPRocky , I see what you are saying, to me it seems odd that a second duration type would be needed.

XAMPPRocky (Jan 23 2020 at 16:03, on Zulip):

I'm not going to argue about it. It's not my design, and I don't particularly care. I'm just providing context.

Nir Friedman (Jan 23 2020 at 16:04, on Zulip):

Do you know if there's more review or discussion of that RFC? I thought such discussions happened on github issues, I did a search there and couldn't find anything

XAMPPRocky (Jan 23 2020 at 16:06, on Zulip):

Nir Friedman (Jan 23 2020 at 16:09, on Zulip):

interesting, thanks. Seems like people are divided over whether it's intended to be used as part of a datetime API or not (or at least, there's 2 people arguing over it).

oli (Jan 23 2020 at 16:09, on Zulip):

I think duration is also non-negative by definition. At least in natural languages you can't say this took "-5 minutes"

oli (Jan 23 2020 at 16:11, on Zulip):

Note that there's also the chrono crate, and there may be others that fit the design you envision better

Nir Friedman (Jan 23 2020 at 16:12, on Zulip):

Again, I don't think analogies (to natural language) are buying you much. If you want to have a duration type that's part of a datetime library, you need to be able to subtract two time_points and have a sensible result, regardless of the order of that subtraction.

Nir Friedman (Jan 23 2020 at 16:12, on Zulip):

If you look at prior art (including Joda), 100% of Duration-like types I've seen in various languages support being negative

Nir Friedman (Jan 23 2020 at 16:13, on Zulip):

I've looked at chrono, IIRC the standard Duration is based on it, but I could be mixed up

oli (Jan 23 2020 at 16:13, on Zulip):

I think it's the other way around

oli (Jan 23 2020 at 16:13, on Zulip):

chrono::Duration may be a reexport of std::time::Duration

Nir Friedman (Jan 23 2020 at 16:13, on Zulip):

Ah maybe you are right. chrono::Duration allows negatives, so that's a start for sure.

XAMPPRocky (Jan 23 2020 at 16:13, on Zulip):

@Nir Friedman Duration is not part of a datetime library. It's only use in std is with SystemTime::duration_since, where it panics if you have a negative duration.

Nir Friedman (Jan 23 2020 at 16:14, on Zulip):

@XAMPPRocky Sure, I agree, Rust's Duration is not currently, it's just not clear from that thread whether that's the intention moving forward or not.

Nir Friedman (Jan 23 2020 at 16:15, on Zulip):

That said, even in that example, I don't think duration_since's behavior is very reasonable (and again, certainly inconsistent with prior art)

Lokathor (Jan 23 2020 at 16:30, on Zulip):

It's a result, not a panic, and the error has a duration value but it's just a "kinda negative" duration.

So really, "duration is always positive" is just weird design all the way through

Nir Friedman (Jan 23 2020 at 16:38, on Zulip):

Yeah. I don't think that "it's reasonably common to only want positive Durations", is reason to limit your main Duration type to just positive. What rust calls SystemTime::now().duration_since(some_other_time) is basically just subtraction of two time points, no matter how you spin it. And you need negative durations for subtraction of time_points to make any sense, and for obvious and desirable identities to hold.

RalfJ (Feb 01 2020 at 14:00, on Zulip):

FWIW when I just wanted to do some simple time stuff, I found C++'s maze of templates incredibly hard to navigate, and was pretty happy that Rust's libstd didn't follow that design.^^ I am sure there is a place for a high-performance lib with a complex API, but I am not sure if libstd is that place.

nagisa (Feb 01 2020 at 15:40, on Zulip):

FWIW Duration is zero cost in a sense that it’s effectively the same thing as the system API (timespec)

Last update: Jun 07 2020 at 10:35UTC