Why Don't Ismailis Perform the Hajj Pilgrimage to Mecca?

The hajj as interpreted by Sunni Muslims is made to the physical Ka‘bah in Makkah, which is revered as the House of God. However, there are three major Qur’ānic facts about the establishment of the physical Ka‘bah as the House of God according to the Qur’an:

• The Ka‘bah was purified by Abraham and Isma‘il (2:125-128) and again by Prophet Muhammad and ‘Ali; therefore, the purity, sanctity and holiness of the Ka‘bah depends upon the purity, sanctity, and holiness of the Prophet Abraham, Prophet Muhammad and the Imams of their progeny, who are the People of the House (ahl al-bayt – declared to be purified by God in Qur’ān 33:33).
• If the Ka‘bah is called Bayt Allah (House of God) in the Qur’an, then the Family of Muhammad are called Ahl al-Bayt (People of the House).
• The Prophet or Imam are those who “raise up” the physical House of God, by God’s permission (iḍn Allāh) as stated in 2:125-128 and 24:36.
• The original Ḥajj, in its Qur’ānic origins, is a pilgrimage to both the physical Ka‘bah and the Prophet of the time and his family. The Qur’an says to Abraham about the original hajj that “And proclaim the ḥajj among mankind: they will come to you (yatūka) on foot” (Qur’an 22:27).

In addition to the Ka‘bah, the exoteric (zahiri) House of God, there is also exists an esoteric (bāṭinī) House of God (Bayt Allāh). According to the Qur’ān, everything in the physical world and the realm of sharī‘ah is an exoteric symbol for a spiritual counterpart or symbolic meaning (mamthūl) in the spiritual world of the ṭarīqah. This is the meaning of the “pairs” that God has created out of all things:

And of all things We have created in pairs, so that you may reflect.
– Holy Qur’ān 51:49

Thus, the physical Ka‘bah has an esoteric (bāṭinī) counterpart without which it possesses no true meaning and for whose sake the Ka‘bah itself is established. The physical Ka‘bah, the physical House of God, is not an immutable structure; it has been demolished, damaged and rebuilt several times throughout history. The physical Ka‘bah is not inherently pure; it has been the site of idol worship for generations and only regained its purity due to the intervention of a Prophet or Imam. Thus, the Ka‘bah cannot be the House of God in the fullest sense because it is inanimate and suffers from both decay and corruption. The Prophet Muhammad, according to several Sunni hadiths, admitted that the physical Ka‘bah is not even built on the original pattern and foundations of Abraham and that he would have demolished and rebuilt it had he not feared the reaction of the Quraysh who were still attached to the customs of jahilliyyah:

Narrated by ‘A’isha: God’s Messenger said to me, “Were your people not close to the pre-Islamic Period of Ignorance, I would have demolished the Ka‘bah and would have rebuilt it on its original foundations laid by Abraham, and I would have built a back door (too).”
– Sahih Bukhari, Chapter 26, Hadith No. 655

The true House of God must be a House which is living, speaking, intellectual and conscious whose essence remains free from decay and impurity. Unlike the realm of ẓāhir which includes corruptible inanimate objects, the realm of bāṭin consists of living realities possessing the fullness of intellect and soul.

Since the very status and existence of the physical House of God is derived from, connected to, and dependent upon the Prophet and Imam, it follows that the Imam of the Time is the esoteric House of God and the Living Ka‘bah. The Imam himself is the inner meaning (bāṭin; ta’wīl) of the physical orẓāhirīKa‘bah. In this sense, the Shi’i Imams have declared:

"We are the Ka‘bah of God ( ka‘batullāh), Exalted is He, and we are the Qiblah of God ( qiblatullāh). We are the Sacred Sanctuary of God ( ḥarāmullah)” 
– Imam Ja‘far al-Sadiq, ( Bihār al-Anwār, 24:211/1, in Muhammad Ali Sabzvari, Tuhfa Yi-‘Abbasi: The Golden Chain of Sufism in Shi‘ite Islam, 68)

The Sufis of Sunni Islam, including Hazrat Jalal al-Din Rumi, professed the very same teaching that the Prophet or Imam of the Time is the living House of God and the spiritual Ka‘bah. Rumi himself writes that the physical Ka‘bah in Makkah is but a branch of the spiritual Ka‘bah – the Prophet and the Friends of God.

"The meaning of the Ka‘bah is the heart of the Prophets and the Friends of God (awliyā’) and the dwelling place of God’s revelation. The [physical] Ka‘bah is a branch of that. If it were not for the heart [of the Prophets and Friends of God] what use would the Ka‘bah be? The Prophets and Friends of God who have totally abandoned their own desires and are following the desire of God. So whatever He commands, they do it."
– Hazrat Jalal al-Din Rumi, (Fihi Ma Fihi, in Arthur F. Buehler, Sufi Heirs of the Prophet: The Indian Naqshbandiyya and the Rise of the Mediating Sufi Shaykh, 43)

The proofs and arguments for the Imam of the Time as the spiritual Ka‘bah and the living House of God are numerous. Firstly, the Qur’ān mentions in the famous ayat al-nūr that the Lamp of God’s Light (nūr) is found: “in Houses (buyūt) which God has ordered to be exalted and that His name is remembered therein; exalting Him in the morning and the evening.” (Holy Qur’ān 24:36). The Imam of the Time is the locus of manifestation (mazhar) of the Light of God and therefore the “Houses” (buyūt) that contain the Light mentioned in this verse refer to the Imams, not to physical houses. This is because only the souls of the Prophets and Imams are pure like the glass of the lamp described in ayat al-nūr and this is why the Prophet himself is called both Light (5:15) and a light-emanating lamp elsewhere in the Qur’ān (33:47). The Imam of the time is the true House of God because the recognition (ma‘rifah) of God is attained through him:

Imam al-Husayn called out to his companions: “God – may His Mention be glorified – did not create the servants except to know Him, and by knowing Him to worship him, and by worshipping Him to be satisfied only by His worship, and to never find satisfaction in worshipping other than Him.” And a man said to him: “O Son of the Prophet, may my father and mother be sacrificed for you. What is this Recognition (ma‘rifah) of God?” The Imam said: “The recognition (ma‘rifah) every people have of the Imam of their Time to whom obedience is due.”
–  (Shaykh Saduq, Ilal al-Sharā‘i’, Vol. 1, 19)

See this article for more details and depth: https://ismailignosis.com/2015/01/02/esoteric-hajj-from-the-physical-kabah-to-the-living-imam/