Apparently so, since the number of buildings you can construct is tied to your construction tech which is 2 buildings per holding per level of construction
was very odd for me as well but in some bookmarks, and some holdings have buildings but a lot are indeed empty
Played my first game yesterday and I was kinda destroyed to a phantom attrition while raiding I didn't really understand but oh well.