As I work on my website project I've been reading today a couple papers by W. Bryan Stolzfus, who was a professor at William Penn College in Oskaloosa who published on tephritids in the 70s and 80s. He wrote a short note describing his finding of a small number of Eurosta solidaginis larvae that lived in goldenrod stems (and successfully emerged as adults) without any visible swelling to the stem.

"The non-gall formers produced a small cavity in the stem pith about twice the size of the larva in the six cavities found."
"From 2,335 non-galled stems collected (Table 1) 16 adults [of E. solidaginis] emerged, nine males and seven females."
"Adult E. solidaginis from non-galled stems were smaller. ... Their body was generally darker but the wings had less darkening of the costal cells. No other consistent differences were noticed."

So, given that finding and your finding of these small galls, I guess there's more variation with this species than first meets the eye.

Stoltzfus, W. Bryan (1989) "A Non-gall Forming Eurosta solidaginis (Diptera: Tephritidae)," Journal of the
Iowa Academy of Science: JIAS, 96(2), 50-51.
https://scholarworks.uni.edu/cgi/viewcontent.cgi?article=1587&context=jias