Generator size and content properties

Six properties replace 37 examples and exercise the size and content of 22 pony_test generator families.
3 files changed+981−629

ponyc #6674 · 7ad061ef23 · pony-test-size-contract

Motivation

Fixed boundary, duplicate, and retry cases run every time. Varying the seed exercises additional sizes and contents. The properties preserve the bounds and identity cases covered by the removed examples.

File Map

Generator properties
Test registration and replaced examples

Test Structure

PropertyCoverage
Gen/size/small_boundsAll 25 ordered bound pairs from 0 through 4 for every generator family.
Gen/size/large_boundsWeighted small and large bounds up to 300, plus exact length 300.
Gen/size/small_domainsDomains of one to six keys, repeated keys, value replacement, and the duplicate stall limit.
Gen/size/extended_boundsFixed legacy inputs, maximum-size sweeps, and the full U8 set domain.
Gen/size/contentScripted sequence order, byte and code-point endpoints, and UTF-32 retry and reset cases.
Gen/size/optional_continuationFixed seed witnesses for the decisions that extend a result past its minimum.

Legacy Test Mappings

All 37 replaced examples
Former testPropertyCase run in every property run
_SeqOfTestGen/size/extended_bounds(0, 10)
_SetOfTestGen/size/extended_bounds(0, 1024) with the full 256-value U8 domain
_SetOfMaxTestGen/size/extended_bounds(0, to) for every to from 1 through 254
_SetOfEmptyTestGen/size/small_bounds(from, to) = (0, 0)
_SetOfMinTestGen/size/extended_bounds(1, 100)
_SetIsOfIdentityTestGen/size/small_domains(0, 100), two forced draws, then stop. Both draws return the same string object.
_SetIsOfMinTestGen/size/extended_bounds(1, 100)
_MapOfEmptyTestGen/size/small_bounds(from, to) = (0, 0)
_MapOfMaxTestGen/size/extended_bounds(0, to) for every to from 1 through 254
_MapOfIdentityTestGen/size/small_domains(0, 100), two forced draws, then stop. Both keys are separately allocated equal strings. The two values differ, and the second replaces the first.
_MapOfMinTestGen/size/extended_bounds(1, 100)
_MapIsOfEmptyTestGen/size/small_bounds(from, to) = (0, 0)
_MapIsOfMaxTestGen/size/extended_bounds(0, to) for every to from 1 through 254
_MapIsOfIdentityTestGen/size/small_domains(0, 100), two forced draws, then stop. Both draws return the same string object. The two values differ, and the second replaces the first.
_MapIsOfMinTestGen/size/extended_bounds(1, 100)
_VecOfEmptyTestGen/size/small_bounds(from, to) = (0, 0)
_VecOfFromToReversedTestGen/size/extended_bounds(10, 5)
_VecOfMaxTestGen/size/extended_bounds(0, to) for every to from 1 through 49
_VecOfMinTestGen/size/extended_bounds(1, 100)
_PersistentListOfEmptyTestGen/size/small_bounds(from, to) = (0, 0)
_PersistentListOfMaxTestGen/size/extended_bounds(0, to) for every to from 1 through 49
_PersistentListOfMinTestGen/size/extended_bounds(1, 100)
_PersistentSetIsOfIdentityTestGen/size/small_domains(0, 100), two forced draws, then stop. Both draws return the same string object.
_PersistentSetOfEmptyTestGen/size/small_bounds(from, to) = (0, 0)
_PersistentSetOfMaxTestGen/size/extended_bounds(0, to) for every to from 1 through 254
_PersistentSetOfMinTestGen/size/extended_bounds(1, 100)
_PersistentSetIsOfEmptyTestGen/size/small_bounds(from, to) = (0, 0)
_PersistentSetIsOfMaxTestGen/size/extended_bounds(0, to) for every to from 1 through 254
_PersistentSetIsOfMinTestGen/size/extended_bounds(1, 100)
_PersistentMapOfIdentityTestGen/size/small_domains(0, 100), two forced draws, then stop. Both keys are separately allocated equal strings. The two values differ, and the second replaces the first.
_PersistentMapIsOfIdentityTestGen/size/small_domains(0, 100), two forced draws, then stop. Both draws return the same string object. The two values differ, and the second replaces the first.
_PersistentMapOfEmptyTestGen/size/small_bounds(from, to) = (0, 0)
_PersistentMapOfMaxTestGen/size/extended_bounds(0, to) for every to from 1 through 254
_PersistentMapOfMinTestGen/size/extended_bounds(1, 100)
_PersistentMapIsOfEmptyTestGen/size/small_bounds(from, to) = (0, 0)
_PersistentMapIsOfMaxTestGen/size/extended_bounds(0, to) for every to from 1 through 254
_PersistentMapIsOfMinTestGen/size/extended_bounds(1, 100)

Mutation Results

791 sites on x86-64 Linux. 660 killed, 129 survived, 2 timed out, and 0 without coverage.

The unfiltered and filtered baselines covered the same 791 sites. Mutant subprocesses skipped the async-action timeout tests. Both full stdlib runs included them.

71 in-scope survivors were equivalent. The other 15 had no in-process kill on this build. All 303 gap-closing kills and the seven former in-scope timeouts failed a property in each of three reruns.

Full lint and the stdlib suite passed in both codegen modes. All 45 counterfactual checks fired.

StatusIn scopeOut of scope
Killed60951
Survived8643
Timeout02
NoCoverage00
86 classified in-scope survivors
LocationCategoryOriginalReplacementClassificationReason
gen_obj.pony:155:23rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:161:36amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:161:49amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:161:58amraddadd_unsafeNo in-process kill on this buildThe only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation.
gen_obj.pony:166:29amraddadd_unsafeEquivalentThe size is below the representable upper bound before the increment.
gen_obj.pony:195:23rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:201:36amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:201:49amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:201:58amraddadd_unsafeNo in-process kill on this buildOnly a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence.
gen_obj.pony:206:29amraddadd_unsafeEquivalentThe size is below the representable upper bound before the increment.
gen_obj.pony:235:23rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:241:36amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:241:49amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:241:58amraddadd_unsafeNo in-process kill on this buildOnly a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence.
gen_obj.pony:246:29amraddadd_unsafeEquivalentThe size is below the representable upper bound before the increment.
gen_obj.pony:320:32rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:320:49rorltneEquivalentThe stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it.
gen_obj.pony:322:32amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:327:54amraddadd_unsafeNo in-process kill on this buildThe only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation.
gen_obj.pony:332:32roreqleEquivalentInserting or replacing an entry never decreases the size.
gen_obj.pony:333:31amraddadd_unsafeEquivalentThe stall counter is below 100 before the increment.
gen_obj.pony:371:32rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:371:49rorltneEquivalentThe stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it.
gen_obj.pony:373:32amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:378:54amraddadd_unsafeNo in-process kill on this buildThe only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation.
gen_obj.pony:383:32roreqleEquivalentInserting or replacing an entry never decreases the size.
gen_obj.pony:384:31amraddadd_unsafeEquivalentThe stall counter is below 100 before the increment.
gen_obj.pony:422:32rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:422:49rorltneEquivalentThe stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it.
gen_obj.pony:424:32amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:429:54amraddadd_unsafeNo in-process kill on this buildThe only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation.
gen_obj.pony:435:32roreqleEquivalentInserting or replacing an entry never decreases the size.
gen_obj.pony:436:31amraddadd_unsafeEquivalentThe stall counter is below 100 before the increment.
gen_obj.pony:474:32rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:474:49rorltneEquivalentThe stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it.
gen_obj.pony:476:32amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:481:54amraddadd_unsafeNo in-process kill on this buildThe only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation.
gen_obj.pony:487:32roreqleEquivalentInserting or replacing an entry never decreases the size.
gen_obj.pony:488:31amraddadd_unsafeEquivalentThe stall counter is below 100 before the increment.
gen_obj.pony:520:23rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:526:36amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:526:49amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:526:58amraddadd_unsafeNo in-process kill on this buildOnly a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence.
gen_obj.pony:530:29amraddadd_unsafeEquivalentThe size is below the representable upper bound before the increment.
gen_obj.pony:558:23rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:564:36amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:564:49amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:564:58amraddadd_unsafeNo in-process kill on this buildOnly a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence.
gen_obj.pony:568:29amraddadd_unsafeEquivalentThe size is below the representable upper bound before the increment.
gen_obj.pony:603:32rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:603:49rorltneEquivalentThe stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it.
gen_obj.pony:605:32amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:610:54amraddadd_unsafeNo in-process kill on this buildOnly a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence.
gen_obj.pony:615:32roreqleEquivalentInserting or replacing an entry never decreases the size.
gen_obj.pony:616:31amraddadd_unsafeEquivalentThe stall counter is below 100 before the increment.
gen_obj.pony:655:32rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:655:49rorltneEquivalentThe stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it.
gen_obj.pony:657:32amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:662:54amraddadd_unsafeNo in-process kill on this buildOnly a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence.
gen_obj.pony:667:32roreqleEquivalentInserting or replacing an entry never decreases the size.
gen_obj.pony:668:31amraddadd_unsafeEquivalentThe stall counter is below 100 before the increment.
gen_obj.pony:708:32rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:708:49rorltneEquivalentThe stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it.
gen_obj.pony:710:32amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:715:54amraddadd_unsafeNo in-process kill on this buildOnly a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence.
gen_obj.pony:721:32roreqleEquivalentInserting or replacing an entry never decreases the size.
gen_obj.pony:722:31amraddadd_unsafeEquivalentThe stall counter is below 100 before the increment.
gen_obj.pony:760:32rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:760:49rorltneEquivalentThe stall count starts at zero, increases by one, and resets to zero after growth. The loop reaches the limit before it can pass it.
gen_obj.pony:762:32amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:767:54amraddadd_unsafeNo in-process kill on this buildOnly a zero-minimum, maximum-USize request can overflow. The bounded probe observed the same empty result with the original and mutant. Unsafe overflow is undefined, so this is not a global equivalence.
gen_obj.pony:773:32roreqleEquivalentInserting or replacing an entry never decreases the size.
gen_obj.pony:774:31amraddadd_unsafeEquivalentThe stall counter is below 100 before the increment.
gen_obj.pony:1225:23rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:1231:36amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:1231:49amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:1231:58amraddadd_unsafeNo in-process kill on this buildThe only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation.
gen_obj.pony:1235:29amraddadd_unsafeEquivalentThe size is below the representable upper bound before the increment.
gen_obj.pony:1256:55amrsubsub_unsafeEquivalentEvery member of the closed ASCIIRange union supplies at least one byte.
gen_obj.pony:1324:23rorltneEquivalentThe size starts at zero and increases by at most one per iteration. The loop reaches the limit before it can pass it.
gen_obj.pony:1330:36amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:1330:49amrsubsub_unsafeEquivalentThe enclosing loop admits only sizes below the upper bound.
gen_obj.pony:1330:58amraddadd_unsafeNo in-process kill on this buildThe only possible overflow requires a maximum-USize preallocation. The baseline cannot complete that allocation.
gen_obj.pony:1336:27rorltneEquivalentThe retry count starts at zero and increases by one after each rejected scalar. The loop reaches the limit before it can pass it.
gen_obj.pony:1344:33amraddadd_unsafeEquivalentThe retry counter is below 100 before the increment.
gen_obj.pony:1356:29amraddadd_unsafeEquivalentThe size is below the representable upper bound before the increment.
43 out-of-scope survivors
2 out-of-scope timeouts
LocationCategoryOriginalReplacementScope
gen_obj.pony:969:50rvrreturn0Out of scope
gen_obj.pony:969:50rvrreturn1Out of scope