The Keep is the most important building in your castle. You may argue that the Barracks is the most important, or a fully leveled-up bakery. The Keep is still the foundation of your castle. It has many functions-- it sets the building speed of your castle, it promotes public order, it stores wood, food, and stone, it prevents a certain amount of resources from being looted. In a battle, it functions as the place where your troops make their last stand. Every part of your castle can be attributed to your Keep and your level.