We model the deposit market, where commercial banks compete using deposit interest rates, and depositors initially distributed among banks, when switching to another bank, bear the exogenous switching costs associated with a lack of information and money transfer fee. We consider both discrete and continuous distribution of depositors over switching costs and find equilibria in pure strategies. The theoretical model in hand allows us to explain the empirically observed negative relation between the size of a bank and its weighted average deposit rate. We show that this dependence may be the result of the history of the banking market formation. Initially, established banks could manage to obtain a majority of depositors with high switching costs, while depositors with low costs could be lost to newly emerging banks. Because of this, previously established banks can set lower deposit rates without fear that their depositors will switch to competitors, and maintain a large share of all depositors in the market. It follows from the analysis that the division of large banks into smaller ones will not lead to an increase in their deposit interest rates, but on the contrary, may even increase discrimination against depositors with high transition costs. It is the reduction of depositors’ switching costs that makes banks to raise deposit rates and thus increase public welfare.