for each combat style (melee, cavalry & missile & so on) there is also an individual tree for markets (rice exchange) and Buddhist monasteries & also recreational buildings like "Sake Dens" & "Gambling halls" the way it works unlike in (Rome or mtw2 most other total war games) is every time you upgrade your castle/town you open up a "slot" for one of these trees, once you have built the first level of Say a recreation building which is "Saki Den" (that slot is now permanently open) so the building of the next level of "Gambling Den" is only contingent on whether or not you can afford it~
Also remember always upgrade your farms first so you have enough food for the expanded settlement if your game grows & you have multiple settlements it makes know difference if a settlement has been upgraded three times but its farming has only been upgraded once as long as other settlements of yours have a surplus example:
Satsuma has a level three castle/town but only a level two farm this equals _-1 but you aslo control Osumi which has only a level two catsle/town but a level three farm which equals +1 all that matters is your cumulative faction total -1+1=0 (this indicator can be found in the very far right bottom of the screen
enjoy!