Thanks to Sammy Kaye Powers for facilitating our “2nd” PHP TestFest planning meeting. We met with Joe Watkins, Sara Golemon, Rafael Dohms, and Nuno Lopes to discuss a plan for how PHP TestFest will track contributions, without burdening the PHP core team.
Here are highlights of our conversation:
I stated that the purpose of PHP TestFest is to introduce PHP developers to alternate ways of contributing to the PHP core.
I outlined a plan for accepting TestFest PRs at a separate repository and providing a website that participants can sign in to (using their GitHub account) to track their PRs and user group affiliation, etc. More details and instructions coming soon; repository is here: https://github.com/phpcommunity/phptestfest-php-src (but don’t make PRs against it until we have written instructions!).
Sara floated the idea of a points-based system for contributors, and we’d use this points system for the prize raffle. This allows reviewers to award more points for meaningful tests. Again, I’ll write up more details and seek more advice before finalizing this.
Rafael suggested that, rather than use the points to award the “big” prizes, we let the reviewers pick the top submission(s) in each category. So, we’d need to come up with categories for that, but some ideas are New Contributor, User Group, etc.
Joe and Sara volunteered to be reviewers. I’d like few more folks who are comfortable with PHP tests and internals to volunteer as reviewers, as well.
Nuno discussed gcov a bit and a few side projects that have been kicked around over the years to parallelize phpt tests. This work is not in scope for PHP TestFest, but it would be great if some wanted to help work on these efforts.
We discussed some of the problems with attempting to track code coverage as a metric. It’s also unclear how we could track “robustness” of the test suite. We can, however, track number of tests submitted, number of “meaningful” tests, and number of new contributors, so these will be some of the metrics we can report on at the conclusion of PHP TestFest.