Generator size and content properties
Six properties replace 37 examples and exercise the size and content of 22
pony_test generator families.
3 files changed+981−629
ponyc #6674 · 7ad061ef23 · pony-test-size-contract
Motivation
Fixed boundary, duplicate, and retry cases run every time. Varying the seed exercises additional sizes and contents. The properties preserve the bounds and identity cases covered by the removed examples.
File Map
Generator properties
Test registration and replaced examples
Test Structure
| Property | Coverage |
|---|---|
Gen/size/small_bounds | All 25 ordered bound pairs from 0 through 4 for every generator family. |
Gen/size/large_bounds | Weighted small and large bounds up to 300, plus exact length 300. |
Gen/size/small_domains | Domains of one to six keys, repeated keys, value replacement, and the duplicate stall limit. |
Gen/size/extended_bounds | Fixed legacy inputs, maximum-size sweeps, and the full U8 set domain. |
Gen/size/content | Scripted sequence order, byte and code-point endpoints, and UTF-32 retry and reset cases. |
Gen/size/optional_continuation | Fixed seed witnesses for the decisions that extend a result past its minimum. |
Legacy Test Mappings
All 37 replaced examples
| Former test | Property | Case run in every property run |
|---|---|---|
_SeqOfTest | Gen/size/extended_bounds | (0, 10) |
_SetOfTest | Gen/size/extended_bounds | (0, 1024) with the full 256-value U8 domain |
_SetOfMaxTest | Gen/size/extended_bounds | (0, to) for every to from 1 through 254 |
_SetOfEmptyTest | Gen/size/small_bounds | (from, to) = (0, 0) |
_SetOfMinTest | Gen/size/extended_bounds | (1, 100) |
_SetIsOfIdentityTest | Gen/size/small_domains | (0, 100), two forced draws, then stop. Both draws return the same string object. |
_SetIsOfMinTest | Gen/size/extended_bounds | (1, 100) |
_MapOfEmptyTest | Gen/size/small_bounds | (from, to) = (0, 0) |
_MapOfMaxTest | Gen/size/extended_bounds | (0, to) for every to from 1 through 254 |
_MapOfIdentityTest | Gen/size/small_domains | (0, 100), two forced draws, then stop. Both keys are separately allocated equal strings. The two values differ, and the second replaces the first. |
_MapOfMinTest | Gen/size/extended_bounds | (1, 100) |
_MapIsOfEmptyTest | Gen/size/small_bounds | (from, to) = (0, 0) |
_MapIsOfMaxTest | Gen/size/extended_bounds | (0, to) for every to from 1 through 254 |
_MapIsOfIdentityTest | Gen/size/small_domains | (0, 100), two forced draws, then stop. Both draws return the same string object. The two values differ, and the second replaces the first. |
_MapIsOfMinTest | Gen/size/extended_bounds | (1, 100) |
_VecOfEmptyTest | Gen/size/small_bounds | (from, to) = (0, 0) |
_VecOfFromToReversedTest | Gen/size/extended_bounds | (10, 5) |
_VecOfMaxTest | Gen/size/extended_bounds | (0, to) for every to from 1 through 49 |
_VecOfMinTest | Gen/size/extended_bounds | (1, 100) |
_PersistentListOfEmptyTest | Gen/size/small_bounds | (from, to) = (0, 0) |
_PersistentListOfMaxTest | Gen/size/extended_bounds | (0, to) for every to from 1 through 49 |
_PersistentListOfMinTest | Gen/size/extended_bounds | (1, 100) |
_PersistentSetIsOfIdentityTest | Gen/size/small_domains | (0, 100), two forced draws, then stop. Both draws return the same string object. |
_PersistentSetOfEmptyTest | Gen/size/small_bounds | (from, to) = (0, 0) |
_PersistentSetOfMaxTest | Gen/size/extended_bounds | (0, to) for every to from 1 through 254 |
_PersistentSetOfMinTest | Gen/size/extended_bounds | (1, 100) |
_PersistentSetIsOfEmptyTest | Gen/size/small_bounds | (from, to) = (0, 0) |
_PersistentSetIsOfMaxTest | Gen/size/extended_bounds | (0, to) for every to from 1 through 254 |
_PersistentSetIsOfMinTest | Gen/size/extended_bounds | (1, 100) |
_PersistentMapOfIdentityTest | Gen/size/small_domains | (0, 100), two forced draws, then stop. Both keys are separately allocated equal strings. The two values differ, and the second replaces the first. |
_PersistentMapIsOfIdentityTest | Gen/size/small_domains | (0, 100), two forced draws, then stop. Both draws return the same string object. The two values differ, and the second replaces the first. |
_PersistentMapOfEmptyTest | Gen/size/small_bounds | (from, to) = (0, 0) |
_PersistentMapOfMaxTest | Gen/size/extended_bounds | (0, to) for every to from 1 through 254 |
_PersistentMapOfMinTest | Gen/size/extended_bounds | (1, 100) |
_PersistentMapIsOfEmptyTest | Gen/size/small_bounds | (from, to) = (0, 0) |
_PersistentMapIsOfMaxTest | Gen/size/extended_bounds | (0, to) for every to from 1 through 254 |
_PersistentMapIsOfMinTest | Gen/size/extended_bounds | (1, 100) |
Mutation Results
791 sites on x86-64 Linux. 660 killed, 129 survived, 2 timed out, and 0 without coverage.
The unfiltered and filtered baselines covered the same 791 sites. Mutant subprocesses skipped the async-action timeout tests. Both full stdlib runs included them.
71 in-scope survivors were equivalent. The other 15 had no in-process kill on this build. All 303 gap-closing kills and the seven former in-scope timeouts failed a property in each of three reruns.
Full lint and the stdlib suite passed in both codegen modes. All 45 counterfactual checks fired.
| Status | In scope | Out of scope |
|---|---|---|
| Killed | 609 | 51 |
| Survived | 86 | 43 |
| Timeout | 0 | 2 |
| NoCoverage | 0 | 0 |
86 classified in-scope survivors
| Location | Category | Original | Replacement | Classification | Reason |
|---|---|---|---|---|---|
gen_obj.pony:155:23 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:161:36 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:161:49 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:161:58 | amr | add | add_unsafe | No in-process kill on this build | The only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation. |
gen_obj.pony:166:29 | amr | add | add_unsafe | Equivalent | The size is below the representable upper bound before the increment. |
gen_obj.pony:195:23 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:201:36 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:201:49 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:201:58 | amr | add | add_unsafe | No in-process kill on this build | Only a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence. |
gen_obj.pony:206:29 | amr | add | add_unsafe | Equivalent | The size is below the representable upper bound before the increment. |
gen_obj.pony:235:23 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:241:36 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:241:49 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:241:58 | amr | add | add_unsafe | No in-process kill on this build | Only a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence. |
gen_obj.pony:246:29 | amr | add | add_unsafe | Equivalent | The size is below the representable upper bound before the increment. |
gen_obj.pony:320:32 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:320:49 | ror | lt | ne | Equivalent | The stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it. |
gen_obj.pony:322:32 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:327:54 | amr | add | add_unsafe | No in-process kill on this build | The only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation. |
gen_obj.pony:332:32 | ror | eq | le | Equivalent | Inserting or replacing an entry never decreases the size. |
gen_obj.pony:333:31 | amr | add | add_unsafe | Equivalent | The stall counter is below 100 before the increment. |
gen_obj.pony:371:32 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:371:49 | ror | lt | ne | Equivalent | The stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it. |
gen_obj.pony:373:32 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:378:54 | amr | add | add_unsafe | No in-process kill on this build | The only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation. |
gen_obj.pony:383:32 | ror | eq | le | Equivalent | Inserting or replacing an entry never decreases the size. |
gen_obj.pony:384:31 | amr | add | add_unsafe | Equivalent | The stall counter is below 100 before the increment. |
gen_obj.pony:422:32 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:422:49 | ror | lt | ne | Equivalent | The stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it. |
gen_obj.pony:424:32 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:429:54 | amr | add | add_unsafe | No in-process kill on this build | The only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation. |
gen_obj.pony:435:32 | ror | eq | le | Equivalent | Inserting or replacing an entry never decreases the size. |
gen_obj.pony:436:31 | amr | add | add_unsafe | Equivalent | The stall counter is below 100 before the increment. |
gen_obj.pony:474:32 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:474:49 | ror | lt | ne | Equivalent | The stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it. |
gen_obj.pony:476:32 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:481:54 | amr | add | add_unsafe | No in-process kill on this build | The only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation. |
gen_obj.pony:487:32 | ror | eq | le | Equivalent | Inserting or replacing an entry never decreases the size. |
gen_obj.pony:488:31 | amr | add | add_unsafe | Equivalent | The stall counter is below 100 before the increment. |
gen_obj.pony:520:23 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:526:36 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:526:49 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:526:58 | amr | add | add_unsafe | No in-process kill on this build | Only a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence. |
gen_obj.pony:530:29 | amr | add | add_unsafe | Equivalent | The size is below the representable upper bound before the increment. |
gen_obj.pony:558:23 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:564:36 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:564:49 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:564:58 | amr | add | add_unsafe | No in-process kill on this build | Only a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence. |
gen_obj.pony:568:29 | amr | add | add_unsafe | Equivalent | The size is below the representable upper bound before the increment. |
gen_obj.pony:603:32 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:603:49 | ror | lt | ne | Equivalent | The stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it. |
gen_obj.pony:605:32 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:610:54 | amr | add | add_unsafe | No in-process kill on this build | Only a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence. |
gen_obj.pony:615:32 | ror | eq | le | Equivalent | Inserting or replacing an entry never decreases the size. |
gen_obj.pony:616:31 | amr | add | add_unsafe | Equivalent | The stall counter is below 100 before the increment. |
gen_obj.pony:655:32 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:655:49 | ror | lt | ne | Equivalent | The stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it. |
gen_obj.pony:657:32 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:662:54 | amr | add | add_unsafe | No in-process kill on this build | Only a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence. |
gen_obj.pony:667:32 | ror | eq | le | Equivalent | Inserting or replacing an entry never decreases the size. |
gen_obj.pony:668:31 | amr | add | add_unsafe | Equivalent | The stall counter is below 100 before the increment. |
gen_obj.pony:708:32 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:708:49 | ror | lt | ne | Equivalent | The stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it. |
gen_obj.pony:710:32 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:715:54 | amr | add | add_unsafe | No in-process kill on this build | Only a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence. |
gen_obj.pony:721:32 | ror | eq | le | Equivalent | Inserting or replacing an entry never decreases the size. |
gen_obj.pony:722:31 | amr | add | add_unsafe | Equivalent | The stall counter is below 100 before the increment. |
gen_obj.pony:760:32 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:760:49 | ror | lt | ne | Equivalent | The stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it. |
gen_obj.pony:762:32 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:767:54 | amr | add | add_unsafe | No in-process kill on this build | Only a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence. |
gen_obj.pony:773:32 | ror | eq | le | Equivalent | Inserting or replacing an entry never decreases the size. |
gen_obj.pony:774:31 | amr | add | add_unsafe | Equivalent | The stall counter is below 100 before the increment. |
gen_obj.pony:1225:23 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:1231:36 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:1231:49 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:1231:58 | amr | add | add_unsafe | No in-process kill on this build | The only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation. |
gen_obj.pony:1235:29 | amr | add | add_unsafe | Equivalent | The size is below the representable upper bound before the increment. |
gen_obj.pony:1256:55 | amr | sub | sub_unsafe | Equivalent | Every member of the closed ASCIIRange union supplies at least one byte. |
gen_obj.pony:1324:23 | ror | lt | ne | Equivalent | The size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it. |
gen_obj.pony:1330:36 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:1330:49 | amr | sub | sub_unsafe | Equivalent | The enclosing loop admits only sizes below the upper bound. |
gen_obj.pony:1330:58 | amr | add | add_unsafe | No in-process kill on this build | The only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation. |
gen_obj.pony:1336:27 | ror | lt | ne | Equivalent | The retry count starts at zero and increases by one after each rejected scalar. The loop reaches the limit before it can pass it. |
gen_obj.pony:1344:33 | amr | add | add_unsafe | Equivalent | The retry counter is below 100 before the increment. |
gen_obj.pony:1356:29 | amr | add | add_unsafe | Equivalent | The size is below the representable upper bound before the increment. |
43 out-of-scope survivors
2 out-of-scope timeouts
| Location | Category | Original | Replacement | Scope |
|---|---|---|---|---|
gen_obj.pony:969:50 | rvr | return | 0 | Out of scope |
gen_obj.pony:969:50 | rvr | return | 1 | Out of scope |