First, "Mu" is the shorthand name of the first koan in a collection called the Gateless Gate or Gateless Barrier (Chinese, Wumengua; Japanese, Mumonkan), compiled in China by Wumen Huikai (1183-1260).
Most of the 48 koans in the Gateless Gate are fragments of dialogue between real Zen students and real Zen teachers, recorded over many centuries. Each presents a pointer to some aspect of dharma, By working with koans, the student steps outside the bounds of conceptual thought and realizes the teaching on a deeper, more intimate, level.
Generations of Zen teachers have found Mu to be a particularly useful tool for breaking through the conceptual fog most of us live in. The realization of Mu often sparks a kensho, or enlightenment experience. Kensho is something like cracking open a door or glimpsing a bit of the moon behind the clouds -- it's a breakthrough, yet there is more to be realized.
This article is not going to explain the "answer' to the koan. Instead, it will provide some background on Mu and perhaps give a sense of what Mu is and does.
The Koan Mu
This is the main case of the koan, formally called "Chao-chou's Dog":
A monk asked Master Chao-chou, "Has a dog the Buddha Nature or not?" Chao-chou said, "Mu!"
(Actually, he probably said "Wu," which is the Chinese for Mu, a Japanese word. Mu is usually translated "no," although the late Robert Aitken Roshi said its meaning is closer to "does not have." Zen originated in China, where it is called "Chan." But because western Zen has been largely shaped by Japanese teachers, we in the West tend to use Japanese names and terms.)
Chao-chou Ts'ung-shen (also spelled Zhaozhou; Japanese, Joshu; 778-897) was a real teacher who is said to have realized great enlightenment under the guidance of his teacher, Nan-ch'uan (748-835). When Nan-ch'uan died, Chao-chou traveled throughout China, visiting the prominent Chan teachers of his day.
In the last 40 years of his long life, Chao-chou settled into a small temple in northern China and guided his own disciples. He is said to have had a quiet teaching style, saying much in few words.
In this bit of dialogue, the student is asking about Buddha-nature. In Mahayana Buddhism, Buddha-nature is the fundamental nature of all beings. In Buddhism, "all beings" really does mean "all beings," not just "all humans." And a dog certainly is a "being." The obvious answer to the monk's question, "does a dog have Buddha-nature," is yes.
But Chao-chou said, Mu. No. What's going on here?
The fundamental question in this koan is about the nature of existence. The monk's question came from a fragmented, one-sided perception of existence. Master Chao-chou used Mu as a hammer to break up the monk's conventional thinking.
Robert Aitken Roshi wrote (in The Gateless Barrier),
"The barrier is Mu, but it always has a personal frame. For some the barrier is 'Who am I really?' and that question is resolved through Mu. For others it is 'What is death?' and that question too is resolved through Mu. For me it was 'What am I doing here?'"
John Tarrant Roshi wrote in The Book of Mu: Essential Writings on Zen's Most Important Koan, "The kindness of a koan consists mainly in taking away what you are sure of about yourself."
Working With Mu
Master Wumen himself worked on Mu for six years before he realized it. In his commentary on the koan, he provides these instructions:
So then, make your whole body a mass of doubt, and with your 360 bones and joints and your 84,000 hair follicles, concentrate on this one word No [Mu]. Day and night, keep digging into it. Don't consider it to be nothingness. Don't think in terms of 'has' or 'has not.' It is like swallowing a red-hot iron ball. You try to vomit it out, but you cannot. [Translation from Boundless Way Zen]
Koan study is not a do-it-yourself project. Although the student may work alone most of the time, checking one's understanding against that of a teacher now and then is essential for most of us. Otherwise, it's too common for the student to latch onto some shiny idea of what the koan is saying that really is just more conceptual fog.
Aitken Roshi said, "When someone begins a koan presentation by saying, 'Well, I think the teacher is saying ...,' I want to interrupt, "Mistaken already!"
The late Philip Kapleau Roshi said (in Three Pillars of Zen):
"Mu holds itself coldly aloof from both the intellect and the imagination. Try as it might, reasoning cannot gain even a toehold on Mu. In fact, trying to solve Mu rationally, we are told by the masters, is like 'trying to smash one's fist through an iron wall.' "
There are all manner of explanations of Mu readily available on the Web, many written by people who have no idea what they are talking about. Some professors of religious study classes in western universities teach that the koan is merely an argument about the presence of Buddha nature in sentient or insentient beings. While that question is one that comes up in Zen, to assume that's all the koan is about sells old Chao-chou short.
In Rinzai Zen, the resolution of Mu is considered to be the beginning of Zen practice. Mu changes the way the student perceives everything. Of course, Buddhism has many other means of opening the student to realization; this is just one particular way. But it's a very effective way.