Retail investors are individuals who buy and sell shares for their own interest and not for any organization. They usually trade in smaller quantities and values.
Stock brokers advise and execute trade for individual and institutional investors and facilitates trade in the stock exchange in return for a fee or a commission. It also negotiates OTC trades i.e. off exchange trades.
Market makers are the ones who quote prices and buy and sell for their own account or for their client accounts. Brokerage houses are a type of market maker.
Institutional investors are institutions that buy large block of funds on behalf of its members and face lower commission and preferential treatment because of their high value transactions.