Reproducible Performance Tests

Kept in a personal, open, repo, run on a company-hosted, cloud, K8S infra.

Folks, I have a crazy thought. Please let me know if you see any flaws with the idea.

Within a company I’m helping, I’d like to persuade our DevOps to:

• Set up a registry for blueprints and/or performance tests.

• So that we could quickly spin up a few machines, and run those tests.

• On our, company-paid, Kubernetes cluster, reproducibly.

At the same time:

• I would like some of those tests / blueprints to be open.

• Ideally, on my GitHub.

• And, of course, I want to be able to run someone else’s open-sources perf tests.

  • On our, company’s, K8S cloud infra.

  • As long as those tests are configured accordingly, of course.

And now, the questions:

• How seamlessly could I integrate the above?

  • Ideally, 100% same build & run scripts.

  • And no extra code.

• Would it be able to use GitHub actions to trigger those test runs?

  • So that the actions are “shared across the company boundary”.

  • I.e., 100% identical GitHub actions, that:

    • take different run arguments, or use different secrets,

    • so that the company is not exposed,

    • but some of my (our!) blueprints / perftests remain open source?

If GitHub secrets can be “overridden” by a forked repo — and I hope that’s how they work — then, I think, this solves the problem neatly. Then I’ve got myself a nice playground for some immediate future.

Bonus points:

• My tests would likely be wrapped into Docker containers.

  • Although, generally, I don’t think they have to be.

  • And perf-testing Docker-run vs. bare metal is also interesting.

• If Docker is used, I’d love the containers to be built from GH actions too.

  • So that no one has to trust “my” containers.

  • They can build their own, on their own hardware, into their own registry.

• Last, but not least: I do want to be able to run these tests on “my” machines too.

  • Via the same set of GitHub actoins.

  • By maybe providing my own AWS / GCP / Azure secrets in GH secrets.

  • Or even pointing that particular run to use my Hetzner instance.

Is this a crazy idea, or might it fly?