In the aftermath of WW II, the British belatedly realised that they had to leave the subcontinent, which had begun to get out of their control through the 1940s. Further Britain deep in wartime debt, simply couldn’t afford to hold on to India and its other colonies.
The seeds of Partition were sown long ago. As early as 1940s, Winston Churchill hoped that Hindu-Muslim antagonism would remain “a bulwark of British rule in India”. The British-supervised elections in 1937 and 1946, which the Congress won easily, only hardened Muslim identity. In the 1946 elections, the Congress Party leaders refused to share power with Jinnah, confident that they did not need Muslim support in order to win a majority vote in elections. These attitudes stoked Muslim fears that the secular nationalism of Gandhi and Nehru was a cover for Hindu dominance. By 1946, Jinnah had managed to present himself as the best defender of Muslim interests in a Hindu-dominated India.
The Congress party claimed that it represented 400 million people. Muslim politicians, Jinnah in particular saw the Congress as a party representing upper caste Hindus and demanded a separate homeland for the hundred million Muslims who were spread across the entire country. At first, Nehru and a few other Congress Party leaders dismissed the idea of Pakistan as a joke. Jinnah ordered mass strikes across India, which morphed into Hindu-Muslim riots. In August, 1946, four thousand residents of Calcutta died within 3 days. It was frenzied violence which spun out of control. Retaliatory killings around the country followed especially in Punjab and Bengal. Gandhi failed with his non-violence approach even with the Congress Party. Many of the congress leaders spoke openly of civil war.
It was under these conditions that Mountbatten arrived in Feb 1947 with a clear mandate to transfer power to the Indians in 15 months. He had to figure out how to transfer power and to whom ! He didn’t have the time to understand the prevailing Indian politics. He slowly started working his way with the key politicians and suggested the Partition. Starting with Nehru other congress leaders like Patel and Rajaji, slowly accepted the idea of Pakistan. Finally even Gandhi has to relent despite his resistance to the idea of Pakistan.
The transition was not easy as several religious and ethnic minorities like the Pashtuns, Sikhs, Baluchis demanded independence. Mountbatten managed to defuse most of the secessionist movements other than Kashmir which had a Hindu ruler over a Muslim majority population. After enough wrangling and some torturous negotiations during May 1947 the Indian leaders agreed for the Partition.
It seems the British saw partition along religious lines as the quickest way to exit. The British were eager to divide and quit and the Indian politicians were too eager to enjoy power.
Had the congress leaders not agreed to Partition, we would have had a civil war (with violence in Punjab and Bengal escalating) and the British were content to be just spectators. We would have had more ethnic strife.
Hence Partition was inevitable. In retrospect, it could have been implemented better.
Very abruptly in June 1947, he announced August 1947 as the date for the transfer of power. No reasons to advance the date was provided. Perhaps he just wanted to run away from India. A London barrister — Clyde Radcliffe was flown to India and just given 40 days to define the political geography of India flanked by 2 wings — West Pakistan and East Pakistan.
No one had prepared for a massive transfer of population. Trains carried nothing but corpses and British soldiers were confined to the barracks. Punjab witnessed ethnic cleansing.
It is a different matter that the 2 nation theory wasn’t realised as envisaged in its original form. With the formation of Bangladesh and many secessionist movements in Pakistan we are bound to witness many more partitions.