The first was the "toolkit" repo that had all the manual maintained dependencies and functions, while the second was a managed repo which would have tags pushed to it by the pipeline.
So basically all the repos had a push hook attached to them that would run this centralized dispatch workflow (on Jenkins, but it could be anything). The dispatch job would sanitize the branch name like joey-b/fancy-feature and attach it to the most recent "released version" + devel timestamp, so you'd end up with like 2.25+20260808-12345+joey-b-fancy-feature, and that would be pushed as a floating tag to the nix-ros repo with all the sources from the hundreds of participating repos either locked to the versions set in the top level 2.25 devel branches or to the specified feature branch, so that a given build could pull together multiple same-named branches from across repos. That would be sent off to Hydra, and nix2container outputs were also built that went to simulator based validation.
But the end UI was pretty nice, basically a nix configuration mapped the names to the those repos so after the build was done you could just pull it and "enter the workspace" environment like:
And if you were hacking on the "base" repo itself, it was very easy to pass flags like --override-input base=ros-base#some-ref or path/to/local for quick iteration.
Anyway, as I say, I'm obviously proud of what was achieved, particularly in a pre-LLM world, and the experience of building this has given me many of pg's blub experiences over the years, where I look at the ways that other build and packaging systems solve these problems and think yes, yes I can see how that works, but also, I have experienced a world where in exchange for some relatively minor strictness tradeoffs, the set of problems that that is solving never had to exist in the first place.
Awesome, thanks for taking the time to put this together, that really is impressive pre-LLM. We wont be at this level for a while but its great to have resources to look at for examples of others have solved things.
https://vimeo.com/767139940
At the time of that talk we threw up a lightly sanitized version of the tooling on github. The basic idea was that there were these two repos:
https://github.com/clearpathrobotics/nix-ros-base
https://github.com/clearpathrobotics/nix-ros
The first was the "toolkit" repo that had all the manual maintained dependencies and functions, while the second was a managed repo which would have tags pushed to it by the pipeline.
So basically all the repos had a push hook attached to them that would run this centralized dispatch workflow (on Jenkins, but it could be anything). The dispatch job would sanitize the branch name like joey-b/fancy-feature and attach it to the most recent "released version" + devel timestamp, so you'd end up with like 2.25+20260808-12345+joey-b-fancy-feature, and that would be pushed as a floating tag to the nix-ros repo with all the sources from the hundreds of participating repos either locked to the versions set in the top level 2.25 devel branches or to the specified feature branch, so that a given build could pull together multiple same-named branches from across repos. That would be sent off to Hydra, and nix2container outputs were also built that went to simulator based validation.
But the end UI was pretty nice, basically a nix configuration mapped the names to the those repos so after the build was done you could just pull it and "enter the workspace" environment like:
And if you were hacking on the "base" repo itself, it was very easy to pass flags like --override-input base=ros-base#some-ref or path/to/local for quick iteration.Anyway, as I say, I'm obviously proud of what was achieved, particularly in a pre-LLM world, and the experience of building this has given me many of pg's blub experiences over the years, where I look at the ways that other build and packaging systems solve these problems and think yes, yes I can see how that works, but also, I have experienced a world where in exchange for some relatively minor strictness tradeoffs, the set of problems that that is solving never had to exist in the first place.