It doesn't happen if you place it on a non-replaceable block, and it doesn't match the advancement criteria. The block the criteria calls for is not being right-clicked, if you place a note block for instance it won't trigger the music note to play here, but it would trigger the advancement for using it.
I would also like to note that for some reason, if you place a block on a replaceable block (i.e. short grass), any advancements that match its item_used_on_block trigger will be activated, despite not having interacted with it
The issue is easily negligible in 24w09a by disabling the damage component (instead of using the unbreakable flag), so I think this post should maybe be marked as "works as intended" now.