DMNTK | Decision Model and Notation Toolkit

Test runner

MIT licensed Apache 2.0 licensed Contributor Covenant

Implementation of the DMN™ Technology Compatibility Kit runner. This runner is provided to help vendors automate test execution for their products, in case they wish to use Rust.

Testing progress

Released versions

| Date | Total | Pass | Fail | Version | |---------|-------|------|------|---------| | planned | | | | v1.0.0 |

Beta versions

| Date | Total | Pass | Fail | Ratio | Version | |------------|------:|-----:|-----:|---------:|---------| | planned | | | | | v0.5.0 | | planned | | | | | v0.4.0 | | planned | | | | | v0.3.0 | | 2023/06/12 | 3101 | 3101 | 0 | 100.0% | v0.2.0 | | 2023/02/10 | 3101 | 3098 | 3 | 99.9% | v0.1.1 | | 2023/01/16 | 3101 | 3098 | 3 | 99.9% | v0.1.0 |

Pre-alpha and alpha versions

| Date | Total | Pass | Fail | Ratio | |------------|------:|-----:|-----:|--------:| | 2023/01/16 | 3101 | 3098 | 3 | 99.9% | | 2023/01/14 | 3101 | 3084 | 17 | 99.5% | | 2023/01/12 | 3101 | 3081 | 20 | 99.4% | | 2023/01/07 | 3101 | 3080 | 21 | 99.3% | | 2023/01/07 | 3101 | 3077 | 24 | 99.2% | | 2023/01/05 | 3101 | 3070 | 31 | 99.0% | | 2023/01/04 | 3101 | 3067 | 34 | 98.9% | | 2023/01/03 | 3101 | 3052 | 49 | 98.4% | | 2023/01/02 | 3101 | 2983 | 118 | 96.2% | | 2023/01/02 | 3101 | 2974 | 127 | 95.9% | | 2022/12/29 | 3101 | 2955 | 146 | 95.3% | | 2022/12/21 | 3101 | 2950 | 151 | 95.1% | | 2022/12/20 | 3101 | 2938 | 163 | 94.7% | | 2022/12/20 | 3053 | 2903 | 150 | 95.1% | | 2022/12/20 | 3053 | 2829 | 224 | 92.7% | | 2022/12/20 | 3053 | 2803 | 250 | 91.8% | | 2022/12/16 | 3053 | 2787 | 266 | 91.3% | | 2022/12/15 | 3053 | 2762 | 291 | 90.5% | | 2022/12/14 | 3053 | 2749 | 304 | 90.0% | | 2022/12/13 | 3053 | 2501 | 552 | 81.9% | | 2022/12/11 | 1966 | 1573 | 393 | 80.0% | | 2022/09/24 | 1966 | 1571 | 395 | 79.9% | | 2022/09/06 | 1707 | 1659 | 48 | 97.2% |

LALR(1) parser

| Date | Total | Success | Failure | Other | |------------------|------:|-------------:|--------------:|---------:| | 2022-01-25 19:56 | 1707 | 1659 97.2% | 48 2.8% | 0 0.0% | | 2021-11-28 13:55 | 1757 | 1706 97.1% | 51 2.9% | 0 0.0% | | 2021-11-12 19:45 | 1757 | 1704 97.0% | 53 3.0% | 0 0.0% | | 2021-11-11 15:14 | 1757 | 1686 96.0% | 71 4.0% | 0 0.0% | | 2021-11-10 19:39 | 1757 | 1681 95.7% | 76 4.3% | 0 0.0% | | 2021-11-09 18:12 | 1756 | 1644 93.6% | 112 6.4% | 0 0.0% | | 2021-11-08 13:51 | 1752 | 1617 92.3% | 135 7.7% | 0 0.0% | | 2021-11-05 21:02 | 1752 | 1616 92.2% | 136 7.8% | 0 0.0% | | 2021-11-03 19:21 | 1752 | 1592 90.9% | 160 9.1% | 0 0.0% | | 2021-11-02 19:38 | 1752 | 1536 87.7% | 216 12.3% | 0 0.0% | | 2021-10-21 20:29 | 1752 | 1461 83.4% | 291 16.6% | 0 0.0% | | 2021-10-15 17:25 | 1752 | 1457 83.2% | 302 16.8% | 0 0.0% | | 2021-10-14 23:28 | 1752 | 1382 78.9% | 370 21.1% | 0 0.0% | | 2021-06-02 20:12 | 1752 | 0 0.0% | 1752 100.0% | 0 0.0% |

LL(1) parser

At this stage of testing, LL(1) parser was not maintainable anymore. Implementing fixes that pass the rest of compatibility tests (7.7%) would require a lot of tweaks in this parser. This version of the parser is also quite slow and requires huge amount of memory.

| Date | Total | Success | Failure | Other | |------------------|------:|-------------:|-------------:|---------:| | 2021-02-15 19:21 | 1752 | 1617 92.3% | 135 7.7% | 0 0.0% | | 2021-02-13 16:59 | 1752 | 1535 87.6% | 217 12.4% | 0 0.0% | | 2021-02-11 20:55 | 1752 | 1507 86.0% | 245 14.0% | 0 0.0% | | 2021-02-10 23:12 | 1752 | 1430 81.6% | 322 18.4% | 0 0.0% | | 2021-02-08 20:48 | 1752 | 1404 80.1% | 348 19.9% | 0 0.0% | | 2021-02-05 20:04 | 1752 | 1373 78.4% | 379 21.6% | 0 0.0% | | 2021-02-04 18:53 | 1752 | 1368 78.1% | 384 21.9% | 0 0.0% | | 2021-02-03 19:25 | 1752 | 1178 67.2% | 574 32.8% | 0 0.0% | | 2021-02-02 18:56 | 1752 | 1154 65.9% | 598 34.1% | 0 0.0% | | 2021-02-01 19:22 | 1752 | 904 51.6% | 848 48.4% | 0 0.0% | | 2021-01-29 21:02 | 1752 | 831 47.4% | 921 52.6% | 0 0.0% | | 2021-01-28 19:19 | 1752 | 830 47.4% | 922 52.5% | 0 0.0% | | 2021-01-24 21:51 | 1752 | 809 46.2% | 943 53.8% | 0 0.0% | | 2021-01-24 20:45 | 1752 | 772 44.1% | 980 55.9% | 0 0.0% | | 2021-01-24 14:03 | 1752 | 754 43.0% | 998 57.0% | 0 0.0% | | 2021-01-23 23:16 | 1752 | 747 42.6% | 1005 57.4% | 0 0.0% | | 2021-01-21 19:27 | 1752 | 739 42.2% | 1013 57.8% | 0 0.0% | | 2021-01-11 19:24 | 1752 | 74 4.2% | 1678 95.8% | 0 0.0% | | 2021-01-06 19:17 | 1752 | 71 4.1% | 1681 95/9% | 0 0.0% | | 2021-01-06 18:41 | 1752 | 50 2.9% | 1702 91.7% | 0 0.0% | | 2021-01-06 13:52 | 1752 | 41 2.3% | 1711 97.7% | 0 0.0% |

License

Licensed under either of

at your option.

Contribution

All contributions intentionally submitted for inclusion in the work by you, shall be dual licensed as above, without any additional terms or conditions.