Stream: t-compiler/wg-rls-2.0

Topic: Runners for custom test annotations


Kirill Bulatov (Jan 22 2020 at 21:05, on Zulip):

There exists a number of custom test annotations: #[async_std::test], #[actix_rt::test], #[tokio::test].

Currently RA does not provide any runnables for those, but at least for those 3 it's the same Cargo command used to run them: you can check it by removing the tokio:: prefix first to force the runneer to appear, run it, return the prefix and rerun the task: the test will run.

Would it make sense to detect those and create runners for them?
We can look for attributes ending with ::test and create a runner for every one found.

Kirill Bulatov (Jan 22 2020 at 21:07, on Zulip):

There are, of course, attributes that require a different command to run the tests, for example, the #[wasm_bindgen_test] one, so we might eventually produce some false-positives, but that seems better than no way to run the tests at all.

Thinking of alternative ways to achieve this, we can add some setting with a list of attribute values to look for to create the runners for them.

std::Veetaha (Jan 22 2020 at 21:44, on Zulip):

To clarify, are you talking about this button over there?
pasted image

matklad (Jan 22 2020 at 23:59, on Zulip):

I think just checking "if it ends with test" should be good enough!

Kirill Bulatov (Jan 23 2020 at 09:30, on Zulip):

To clarify, are you talking about this button over there?

Yes, this is what I want to have for various #[whatever::test] annotated methods.

I think just checking "if it ends with test" should be good enough!

Great, my thoughts also.
I'm not entirely sure how to extract the info I need from the ast::Attr, but I'll work on that and find a way hopefully.

Wojciech Polak (Jan 23 2020 at 09:54, on Zulip):

Hey, I'm maintainer of test-case (https://github.com/frondeus/test-case)

My macros are #[test_case(...)] which are expanded into one or more functions with #[test] attribute.
It'd be great if RA could support not only .*test but also test.*. Maybe .*test.*?

Kirill Bulatov (Jan 23 2020 at 10:09, on Zulip):

Sounds reasonable, I'll try to come up with a better heuristics.

std::Veetaha (Jan 23 2020 at 10:18, on Zulip):

I bet we can benefit from the experience of other tools. For example, I know that Java extension provides an ability to run and debug a single JUnit test from VSCode. I don't believe that extension tries to heuristically determine tests, but rather I suppose they just integrate with major test frameworks.

Florian Diebold (Jan 23 2020 at 10:25, on Zulip):

the long-term approach would be to expand the attribute macros and detect test cases in the expanded code, but we're not there yet

Kirill Bulatov (Jan 23 2020 at 10:32, on Zulip):

I think waiting for the macros expansion would be the best way, since it's more generic.

I've had a quick glance and the way the JUnit tests are supported in the RedHat plugin and it looks like the VSCode extension needs another one to run the tests: https://github.com/redhat-developer/vscode-java/blob/master/CONTRIBUTING.md#b-1-setting-up-the-jdt-language-server-in-vs-code
and they call it by simply hardcoding the logic and assuming that any test is a JUnit test: https://github.com/testforstephen/vscode-pde/blob/master/src/extension.ts#L76 (also see the isTestMethodNode method).

So, it's more or less as simple as my heuristics now :)

std::Veetaha (Jan 23 2020 at 11:01, on Zulip):

Hmm, so does it mean that they support only JUnit test framework?

Kirill Bulatov (Jan 23 2020 at 11:20, on Zulip):

That is how I've understood from a quick look at the sources.
But don't believe me, check it yourself, I've posted the related links above.

Last update: Sep 22 2020 at 02:15UTC