I created a quick grout texture within Substance Designer sticking to my initial reference. Here’s the height map for it.
I then used this as a mask in zBrush to inflate details on a plane behind the bricks.
Lastly, I started exporting a few first test of the actual texture passes. I will tweak it more tomorrow and export things like cavity and AO as well.