Introduction
Even a cursory look at the Qur'an will show that the Muslim is asked to follow Allah and the Messenger. "And obey Allah, and obey the Messenger"is an oft repeated statement. One verse says that obedience to the Messenger is equivalent to obedience of God. [ Q 4:80].
The logic behind this is quite simple, Muhammad, Sall Allahu 'alayhi wa sallam, was the final Prophet. He is also a Rasool that is sent for the benefit of humanity. He served as God's representative on Earth. [Q 33:40, 7:157]
While the Messenger of God was alive, the method of obedience went without question. He was there, if questioned he could give the answer, if he says something needs to be done, the believers were obligated to do it. How can these commands be obeyed after the Prophet's death?
Birth of the Hadeeth
That the Prophet's statements, actions and that which he sanctioned [ Aqwaal, Fi'l, Taqreer] would become a source of law and reference is quite natural being that he was representing the Divine. Upon the Prophet's passing, the Muslims expanded, going into new areas. Many of the companions of the Prophet [Sahaabah, Radee Allahu 'anhum], likewise, moved to those areas, finding themselves [due to various reasons] in places as diverse as Iraq, Syria, Arabia, and Egypt. Wherever they went,they took their memories of Rasoolullaah with them. They would teach students.
Inevitably, sayings on all sorts of issues, serious and mundane, entered the public sphere attributed to the Prophet, upon whom be peace. This situation led scholars of hadeeth to form methodologies to ascertain as much as possible what was indeed authentic. This is the summary of the science of 'Ilm ar Rijaal, which focused on the lives and character of the narrators, the chain of transmitters linking the particular assertion to the Prophet.
As an example of the diligence the Hadeeth scholars went through, it is said that Imam Muslim [ d. 875 C.E.], the compiler of the collection known as Saheeh Muslim, examined 300, 000 reports, but only included 4,000 in his collection, excluding repetitions.
Hadeeth methodology is important, but not perfect
The flaws in making character judgments on people in the chain of transmitters and in evaluating the authenticity of a particular report [matn] are obvious. The classifications such as weak [Da'eef],Mawdoo' [false], Hasan [Good], Saheeh [authentic] etc that entered this science are attempts to deal with this situation, but no authorities has ever said that the processes are error free.
The fact that we have different Hadeeth collections is an indicator of this, Imam Muslim would accept a narration not found in Imam Bukhari's collection, and vice versa. Indeed, the phrase Muttafaqun 'alayh, found in many anthologies, shows when a particular report is accepted by both Hadeeth Imams.
In the Sunni tradition, Bukhari and Muslim's collections are considered the most authentic and respected, although other collections do exist and are likewise consulted. This is also an indicator of the freedom of intellect that existed in those days.
Qur'an alone?
In recent years, we have seen a rise of a trend to reject Hadeeth altogether, to form an Islam that does not use hadeeth at all as a point of reference. This trend has been widely viewed by the scholars with suspicion, many scholars actually making Takfeer on what they call "Hadeeth rejectors"[Munkar-e- Hadeeth in Urdu], declaring them totally out of the pale of Islam. This trend has emerged due to a number of factors, among them are the fact that some narrations attributed to the Prophet are highly questionable both ethically and scientifically
I think this trend to have a "Qur'an only" [or "Quranite"] Islam, while sounding attractive, creates more problems than it solves. Indeed, among the Qur'an alone supporters one finds strange divisions and a rather incoherent philosophy. There are Quranites who argue that there is no obligation for prayer in the Qur'an, that the Qur'an actually allows alcohol, and a host of other things that, should the wide majority adopt, would actually make Islam meaningless.
Qur'an and Hadeeth together?
The trend to treat Hadeeth as equal in value and authenticity to the Qur'an is also dangerous. The Qur'an exists as a living document for the scholar and the lay person to consult. It has been in continuous usage since its revelation, and, most importantly, it has been declared by Allah to be under his protection [Q 15:9].
Moreover, many things are said in our nations and mosques by persons here or there as "Sunnah" but are actually not Sunnah at all. People have a tendency, especially in the world of religion, to say things, to quote things without actually knowing for certain its source.The listener [and often the speaker] doesn't have the resources or the ability to ascertain the truth of the alleged 'hadeeth'.
For the people of knowledge, the teachers and the sincere workers of the Islamic religion, they must study both the Qur'an and Hadeeth. They must delve into the world of texts, studying its intricate details from the disciplines in which they have strength. I must say here I have great respect for Nouman Ali Khan, head of Bayyinah institute, famous for his delivery of the Quranic message, when he recognizes his weaknesses. He will not answer questions on Hadeeth or Fiqh, because- as he says- that's not his field. He sees himself as still a student of those fields, and doesn't wish to visit his own confusions onto his audiences.
For lay people
For Non scholars, I actually don't recommend study of the hadeeth literature, at least not all the volumes of Bukhari, Muslim, Tirmidhee and the other collections. These volumes are for the scholars, not the average people. A doctor is a doctor, he may read and study books on medicine from time to time because that's his job. That's his field. A novice simply looking up something and making a judgement may be doing more harm than good. The information may be outdated, it may be incorrect, it may need to be applied in some other context unknowingly omitted, and on and on.
The great scholars and spiritual masters in our tradition have recognized this. Thus, they compiled anthologies containing statements, guidance that all could use and benefit from. Among them are those which we highly recommend, such as Forty Hadeeth [compiled by Imam An Nawawi, d. 1277 C.E.], Riyadhus Saaliheen, also compiled by Imam Nawawi, Bukhari's book of morals and manners.
The Prophetic Sunnah is well-known, and is highlighted throughout the Qur'an itself [Ft.1]. The details of his life are accessible for all. I highly recommend The Life of Muhammad by Muhammad Husayn Haykal.
Footnote
[1] This is elaborated a bit more in "Muhammad: The Prophet of Islam. http://shamsuddinwaheed.blogspot.com/2012/02/muhammad-prophet-of-islam-dispelling.html