We had a suite of thousands of automated tests that we developed in parallel during feature development. People were assigned to write those, e.g. "Test that if one uses such-and-such language construct in this way, that the resulting abstract syntax tree then contains the following annotation..."

Those automated tests were essential, and they wrote either PASS or FAIL messages to stdout. So all I had to do was write a program to scrap stdout for PASS and FAIL, and add the test ID and result to a relational database.