You cannot scroll down in the Combat section of the creative inventory

Confirmed.

That's because there is nothing down there, the set of items fits the window perfectly. The reason you see more books in the search is because some are coming from the tools section.

There are 8 or more rows with books tagged with "Combat".
Search for "book" in the inventory and you'll see.

Maybe this is intended that only the highest levels are shown in at the specific tabs and the "rest" of the books can only be found by searching.

Yeah, I'm pretty sure that's intended. None of the other tabs with enchanted books have any besides the max level ones and all of the combat books found by searching have their maximum level book in the combat tab.