'Writing as a Visual Art'
by Graziella Tonfoniwith James E Richardson with a foreword by Marvin Minsky
The ability to learn to read and write is still acknowledged universally as one of the human mysteries. The author encourages us to equate writing with concepts of shape, texture and other visual concepts - stimulations that are recognised as an important way for meaning and value to be imparted to others.
Foreword by Marvin Minsky
"Text is the self-portrait of human thought."Ted Nelson
We all know how to listen and speak. We're all fairly good at conversing. We take it for granted that when people write, the readers will easily understand. Language involves remarkable skills, yet we rarely ask how such things work. Why are we able to read at all, when written words are mere physical marks, whereas thoughts and ideas are not things at all. Ignoring this, we simply declare that eloquent authors have talents or gifts, as though they either were born that way or were raised to those heights by some magical hand.
In truth, it's quite the opposite. Language skills don't come easily; they develop through years of trying to speak, from our earliest stages of childhood. We work at this both vocally, and privately inside our minds. Yet we never consider that conduct as work; instead we regard it as more like play, forgetting that play is our hardest work. This is the paradox of human psychology: the things that we find the most easy to do are often the ones the most hard to explain. Of course this isn't always true, but it makes an excellent story.
Consider those mental monologues that most of us experience. Few other kinds of mental work engage our brains so constantly. Vision, despite its complexity, appears to need no thought at all. It seems equally easy to stand or walk, yet these are also remarkable feats. We also do commonsense reasoning, and react to social stimuli, while scarcely aware that we're doing those things. How could such complex activities seem so simple and spontaneous? Perhaps because all that work is sealed off in separate systems of the brain, so that each one's enormous complexity would not disrupt the other ones. That could be why we're unaware of how we see, or hear, or think: none of those systems knows or cares how any of the others work.
"Be careful of reading health books; you may die of a misprint."Mark Twain
Language is our most powerful way to tell other people about our ideas. Yet although this is a wonderful thing, we know almost nothing of works. What really are those "talents and gifts" that writers use to communicate? We almost never think about this because it all seems so natural. Nor do we have the faintest idea of how we choose words and make sentences. Yet every speaker knows many ways to describe and explain, to indicate and illustrate, to elaborate and reformulate. We're all fairly good at sentences. Less common are the skills one needs for writing (or reading) good paragraphs. And only a small minority are proficient at such larger forms as chapters, essays, or major reports.
Deficiencies in writing skills can leave a reader wondering, "I can't figure out why they're telling me this." But it isn't always the writer's fault when a paragraph is misunderstood; this also happens when readers are weak at reading the signs that writers use to indicate the functional roles that each of their statements is meant to serve.
We already remarked that language skills are developed through years of hard work. This idea could be discouraging to those who seem less talented, for it seems to imply that any improvement would require too much exertion and pain. But although this is the prevalent view, perhaps our culture has exaggerated those differences, in the course of magnifying the virtues of our heroes. We don't know how to account for those prodigies of eloquence. But once we make ourselves recognize that we we're no better at explaining what common folk do, those differences seem less daunting.
To put some flesh on that argument, here's an arithmetical metaphor: suppose that a good explainer must be a master of precisely one thousand language skills. If so, then a person who lacked ten of those would be missing only one percent. Yet if that were enough to make something go wrong with one of every hundred words, then one such error will likely occur in almost every paragraph. Of course I don't mean this to be taken literally. We mathematicians explain things best by ultrasimplifying them. This "example" is only intended to show how a very small deficiency could produce an effect that might appear to depend on a much larger factor.
There are people who are not so proficient with words, yet are brilliantly clear with diagrams. There are others whom we can understand, only when they wave their hands. They use other forms of rhetoric that are usually not embodied in text, and perhaps those skills live in parts of their brains that are not directly concerned with words.
"I have never had a thought which I could not set down in words with even more distinctness than that which I conceived it."Edgar Allen Poe
Socrates: Anyone who leaves behind him a written manual, and likewise one who takes it over from him, on the supposition that such writing will provide something reliable and permanent, must be exceedingly simple minded.
Author: But if something is written clearly enough, we can usually understand what it says.
Proust: That's what we think is happening, but each reader reads only what is already inside himself. A book is only a sort of optical instrument that the writer offers to let the reader discover in himself what he would not have found without the aid of the book.
Author: Well, I agree that a reader has to know what the separate words mean, but certainly you can say something new in how those words are arranged.
Socrates: Perhaps so, but remember that writing is really no better than painting: you can never be sure what will come from them. The painter's products stand before us as though they were alive: but if you question them, they maintain a majestic silence. It is the same with written words: they seem to talk to you as though they were intelligent, but if you ask them anything about what they say, from a desire to be instructed, they go on telling you just the same thing forever! And once a thing is put in writing, the composition, whatever it may be, drifts all over the place, getting into the hands not only of those who understand it, but equally of those who have no business with it; it doesn't know how to address the right people and not address the wrong. And when it is ill-treated and unfairly abused it always needs its parent to come to its help, being unable to defend or help itself!
Author: But once the writing is well understood, the result inside the reader's mind is not like a passive document. Good language builds active structures in a listener's brain, and then those structures can interact with other things that the listener knows.
Critic: That's easy to say, but what does it mean? Given that texts are mere strings of words, how can they build, in readers' minds, structures that can interact. You have to say more on how structures are built, and more about how they are used.
Author: Presumably, we build them in much the same way that we build anything else. First we have to activate some units that already exist, somewhere inside the reader's brain. Then we must make new connections between them, and finally we can turn them on. That's how anything new is made, no matter whether it is composed of computer parts or animal cells.
Critic: That's really too general to be useful here, where we're speaking of language and what it means. Specifically, what are those units made of, and how do we connect them so they can work in meaningful ways?
Author: No one yet knows the answer to that. The most plausible current hypothesis is that the units are assemblies of nerve-cells, and these get interconnected by enhancing transmission through synapses.
Critic: That's too low a level to be useful to me. I'd like a higher level example of how a sentence could cause a listener's brain to build an appropriate structure.
Author: OK. "John drove Mary from Boston to New York." As soon as the sound of "drove" was sent to the language centers of your brain, it was identified as a certain form of transportation verb. This then activated an already existing structure that functions as a simple sort of story script. By itself, that script represents only the vague idea that someone will cause something to be moved from one place to another. But the other words are swiftly applied to replace those generalities by more specific substitutes. Now tell me where John's trip began?
Critic: Obviously, in Boston.
Author: How do you know that?
Critic: Because you said 'from' Boston.
Author: And where did it end?
Critic: In New York, because you said 'to' New York. We use prepositions like 'from' and 'to' to indicate which items should be attached to which vacancies in the story script.
Author: Who did the driving?
Critic: There was no preposition for that, but it had to be John because that name came just before the word "drove." We also use the order of words to indicate how to fill that slot.
Author: How did you know that John drove a car, rather than a truck or a bus? That wasn't in the words at all. In fact the sentence said "John drove Mary". How did you know that John didn't drive Mary.
Critic: It's a matter of everyday common sense that we rarely mention the obvious. We assume by default that John drove a car, and we never drive a person. (It would be a different story indeed, if we knew that John's car was named 'Mary'.) As for him driving a truck or a bus, we'd expect the writer to mention it if the vehicle were very unusual.
Author: Precisely. Although this seemingly simple sentence expresses many relationships, no one has trouble with connecting them up. This is because we all are masters of using those transportation story scripts. Each particular action-verb activates a already learned representation. In the case of transportation verbs, each script is composed of three descriptions: of how things were before the action, after the action, and the path or trajectory during the action. It is because we all learn the same rules for using such representations that we all can agree on how to assign which terms to which "roles" in all three sub-descriptions.
"Words are only painted fire; a book is the fire itself."Mark Twain
Critic: You're talking about what people call the 'rules of grammar.' But how do we know how to use them?
Author: It takes most children several years to learn to do it properly. Some researchers have maintained that some of those skills are built into our brains from birth. But I suspect that this seems so, largely because we learn those skills in those early years from which we have few memories.
Critic: But if grammar is learned, and not innate, then why are all grammars so similar in all societies?
Author: Because the forms of the grammars are strongly constrained by the forms of the structures that people for representing what we learn. I suspect that those structures are what is largely innate. For example, I'd be willing to bet that our brains are born with pre-wired hardware that makes it easy to represent actions as pairs of frames with trajectories. All cultures soon discover this and, consequently, all languages come to use such verbs, with somewhat similar grammar rules. And that, in turn, might be the cause of why all cultures have sentences.
Feynman: There was a sociologist who had written a paper for all of us to read. I started to read the damn thing, and my eyes were coming out: I couldn't make head nor tail of it. Finally, I said to myself, "I'm gonna stop and read one sentence slowly." So I stopped at random, and read the next sentence very carefully. 'The individual member of the social community often receives his information via visual, symbolic channels.' I went back and forth over it, and translated. You know what it means? 'People read...'
Author: You can certainly make sentences hard to read by filling each slot with irrelevant stuff.
Critic: And using pretentious or obsolete terms like 'via' for trajectories. But where is all this going? Those grammar rules apply only to very small fragments of text.
Author: Precisely the subject of this book. Grammar constrains our sentences, but we do not have such neat constraints when it comes to larger spans of text. When what you want to describe does not fit into a single sentence, you may be able to say it in two or three sentences, but then you'll also have to indicate to your reader how to interconnect those two or three meanings.
Critic: And that's exactly what you just did. Your last sentence was composed of three separate sub-sentences, centered on the verbs 'does,' 'say,' and 'tell.' You connected them with the two conjunctions 'then,' and 'but.'
Author: And see how we use that neat word "but." It tells you that there was something wrong with what you recently understood. It instructs you to go back and replace or augment some structure that you have assumed by default.
Critic: True, but now you're speaking psychologically. Ideas like that lie far outside the scope of traditional grammar rules.
Author: Yet those are things we have to know in order to understand real text. In my opinion, the very concept of grammar is perverse because it suggests that we can understand what language is without good ideas about how it works. I suspect that this is largely why we still know so little about how language means.
Critic: I have to agree with much that you've said. Sentences are strongly constrained by grammar rules, while paragraphs are a good deal more free. Then, when it comes to larger forms, there are scarcely any rules at all, but only individual styles. The problem is that so many writers are badly deficient in style.
"...As a child, I became aware, somehow, that what I put down on paper was, at least some of the time, exactly what I wanted to say."Jeremy Bernstein
"Writing as a Visual Art" asks why so many people can't write good text. Their problems arise on every scale, from filling each sentence with suitable words, to choosing the name of a book. Few people have trouble understanding simple explanations. But we all have trouble with both writing and reading, when it comes to more complex texts. Let's look at some problems on each scale of size.
Selecting the right word can be troublesome, but most people know how to use most of their words. Some people have unusually large vocabularies, and are more able to select 'just the right word.' But if that capacity overgrows, it then can do more harm than good. A scholarly friend of mine reads each new dictionary, and prides himself on being able to find the exactly appropriate words. Unfortunately, no one can understand his books, because of not having seen those words before.
Most people have little trouble with arranging words into sentences. However, a sentence that is grammatically "correct" can still be difficult to understand, if the writer has tried to use it to combine too many different kinds of ideas. In such a case it's better to break up the argument into several separate sentences, to form a paragraphic group. But that can lead to trouble, too, unless we mark each sentence or phrase with some device to indicate how it relates to other parts.
More serious problems are bound to arise with texts of larger scales of size. When a writer explains more complex things, how is the reader supposed to know what each part of the text is trying to do? When writing or speaking, we tend to lose sight of some of our current purposes. I often find while I'm lecturing, that some part of my mind is wondering if I'm repeating a point for emphasis, or trying to say something new? Am I attempting to summarize, or starting to reformulate, or merely making a small augmentation in something that came before? Am I clarifying, or exemplifying, or describing evidence for a statement already made, or am I preparing the case for a point subsequently to make. Frequently it is none of those; instead, writers often use other techniques to induce a reader to favour some view without being explicitly told to; this can go so far as to recount entertaining stories that have little to do with the argument, but, by engaging the reader's sympathy or ! interest, make the essay more pleasant to read.
Such tactics can be valuable, but only if the reader's mind knows what to do with each fragment of text. When such devices fail to work, and therefore some of the readers get lost, we can't conclude that the writer has failed to communicate. It could be quite the opposite, if the writer's confusion is transferred directly across to the hapless reader's mind. Indeed, I fear I've done this now. I had intended, from the start, to focus on one clear idea -- that the goal of good writing is finding good ways to copy things from one mind to another. Instead of clearly explaining this, I find myself rambling aimlessly. If I had taken Tonfoni's advice, we'd surely be finished by now.
"If what we were discussing were a point of law or of the humanities, in which neither true nor false exists, one might trust in subtlety of mind and readiness of tongue and in the greater experience of the writers, and expect him who excelled in those things to make his reasoning more plausible, and one might judge it to be the best. But in the natural sciences, whose conclusions are true and necessary and have nothing to do with human will, one must take care not to place oneself in the defence of error; for here a thousand Demosthenes' and a thousand Aristotles would be left in the lurch by every mediocre wit who happened to hit upon the truth for himself."Galileo Galilei
Critic: And what, precisely, is her advice?
Author: That marvellous as writing is, it can be improved with visual aids.
Critic: You mean, messing up a manuscript with drawings and diagrams?
Author: Not really. Small changes can make large differences. A little can go a long way. We can do a lot with only a few graphical symbols.
Critic: Yet although publishers have long had such capabilities, most good writers don't seem to need them. You don't find any diagrams in the most popular novels.
Author: That's because in popular novels, the reader already knows what's going to happen.
Critic: Nonsense. They're always full of surprises.
Author: Yes, in many minor respects. But in general literature, there are only a few basic large-scale plots, and every good reader knows all of them. When writers don't conform to them, their stories do not become popular.
Critic: I don't see anything wrong with that. Familiar forms make understanding easier. And inside those constraints, good writers can still do almost anything they want to do.
Author: That may be true, for conventional literature. But it works less well for describing things that do not conform to conventional plots. The structural schemes that work for stories don't always work for real things. I have often spent many hours failing to understand a few pages of what appeared to be a well-written text, only to find that a few minutes of talk with an expert made everything clear. The trouble comes, as Socrates said, when the text won't answer your questions.
Critic: Then perhaps those pages were not so well written.
Author: That's often the case, but other times the trouble goes deeper than that. When you have to explain something intricate, then writing, without any visual aids, can simply be too linear. An author can make any number of points, but sometimes the connections must cross one another too many times. This isn't much of a problem in stories, because time itself is linear. But when you're explaining a complex machine, or a fragment of higher mathematics, or the workings of an organization, you need to be able to jump out of time so as not to get stuck on a single track.
Critic: That makes sense, but I have another objection. The use of graphics might bring more loss than gain. How often even a well-made film lacks effects that were in the original book, because imposing definite images constrains the viewer's imagination. After all, no writer knows all that those readers may know. Therefore you want to leave plenty of room to exploit the reader's knowledge and imagination.
Author: True, but Tonfoni is not proposing to augment books that already serve well their purposes. She's trying to help us in other domains where conventional methods don't work so well. And, yes, using these methods might indeed constrain the reader's imagination, but that isn't always a bad thing to do. When you have to explain a complex thing, you may need to maintain more control.
Critic: Granted. But we often do that anyway. Even the writers of unadorned text use graphical groupings in any case. We rarely see texts that are not divided into headings, sections, and other such shapes.
Author: True, but still those are essentially linear. Contrast such writing with face-to-face conversation. Then you usually find yourself engaged in many different activities, all at the same time. You use certain gestures to put the other person at ease. You invent other gestures at various points, to be later re-used as reminders. You modulate your tones of voice, and animate your head and face; all these devices help guide the listener to emphasize and reorganize internal structures and processes.
Critic: You can also use such gestures to distract the listeners' attention or, by appearing friendly or conciliatory, you may be able to gain approval of your proposals even when the arguments are weak. You can do similar things with writing too, like inventing your own critic, so that you can win all you arguments without regard to their merits.
Author: In any case, it really seems miraculous that linear writing can work at all. This is because writers have been making stylistic inventions for thousands of years. Still, many texts are simply too hard to read, and the point is that we can make our meanings much clearer by adding just a few graphical devices.
Critic: Fine, but what does meaning mean. That is a concept that the finest philosophers have failed to clarify.
Author: That's mainly because of trying to describe what meanings are, instead of how they are used. That's precisely the wrong way to go. We consider something to be meaningful, not when it happens to have certain particular characteristics, but when we find ourselves able to apply it to, or use it with, a variety of processes. When I say that an expression seems meaningful, I'm saying more about myself than about that expression.
Critic: That sounds too fuzzy to be useful.
"An idea with a single sense can lead you along only one track. Then, if anything goes wrong, it just gets stuck, thought that sits there in your mind with nowhere to go. That's why, when a person learns something "by rote" Q that is, with no sensible connections Q we say that they "don't really understand." Rich meaning-networks, however, give you many different ways to go: if you can't solve a problem one way, you can try another. True, too many indiscriminate connections will turn a mind to mush. But well-connected meaning-structures let you turn ideas around in your mind, to consider alternatives and envision things from many perspectives until you find one that works. And that's what we mean by thinking!"Marvin Minsky
The meaning of a word or phrase depends first on how we represent it in our minds, and then on how we've connected that to all the other things we know. If we link it to only one other thing, then virtually nothing will be gained. That's why it's almost always wrong to seek the "real meaning" of anything. A thing with only one meaning has scarcely any meaning at all.
Critic: Isn't there something wrong with that? It isn't enough just to make connections, so that every idea is based on several others. Don't you have to start somewhere, with more solid things? Unless your network of ideas is grounded on some concrete foundation, it will all float up in the clouds.
Author: There is nothing wrong with a group of thoughts in which each lends meaning to the rest -- the way that each strand in a rope or a cloth keeps the other ones from falling apart.
Critic: Nice metaphor, but the rope won't work without those strands. Anyway, I wasn't looking for and endless philosophical argument. How does all of this connect with the subject of Tonfoni's book?
Author: Through the idea that meanings themselves are usually multi-media. For example, what happens in your mind when I say just the one word, 'telephone?'
Critic: It leads me to think in several ways. Some of these are visual. I can think of a telephone's physical form, perhaps dumbbell shaped, with a wire attached. Although most of the newer cordless ones are wireless, with more box like shapes. I also have kinesthetic ideas about holding the phone against my ear with the microphone close to my mouth. I can imagine holding it either with my hand, or by clamping it between shoulder and chin. ( Those newer phones don't do this well; they either fall off, or my face presses buttons that make it beep. ) Also, 'telephone' evokes social thoughts; a telephone is an instrument of spoken communication. I haven't called my mother lately. Economic thoughts, too. Last month came a huge long distance bill last. Visual, kinesthetic, social, emotional, economic; isn't it amazing how a single word can engage so many different realms of thought. But how does all of this apply to texts as compared to telephones?
Author: Each of those realms requires you to use several different kinds of brain-representations. So your 'concept' of a telephone is likely to involve at least a dozen different types of networks in your brain. No one yet knows what those are like, but it seems plausible that some of them might resemble what computer scientists call semantic networks. Other brain centers might employ various forms of sequential scripts; others might use fuzzy pattern matching, while yet other brain mechanisms construct various sorts of symbolic-image representations. The point is that every reader of text must be do many of these sorts of things, some of which may lead to problems.
Critic: What sorts of problems do you have in mind?
Author: When language builds things in a listener's mind, it is easy to indicate localized links. But when thoughts that a writer wants you to connect get further apart in the serial text, it gets hard to find words to connect them. Pronouns work well for nearby things, but are clumsy when skipping a paragraph.
Critic: You can get around that by naming something at one place, and using that name at a later point.
Author: But when you do too much of that, it overloads your reader's verbal memory. You can add more links with less mental strain by using visual symbols and graphical links.
Critic: Are you saying that we have more visual than verbal memory?
Author: It isn't that either one is larger or better, but that they engage different kinds of machinery that we can use in different ways. It is very hard to understand a verbal description of a system or gadget with five or six parts, yet that's easy to do with a diagram. Why shouldn't it be just the same for exposing the structures of arguments?
Critic: Then why didn't Shakespeare need diagrams, or Moliere, or Melville?
Author: Melville could have told us more about those nautical contrivances if the publisher had given him more latitude. Imagine how hard it would be to write about mathematics -- or any other technical subject -- if you couldn't use those special effects, symbols, equations, and stuff like that.
Critic: If mathematicians gave those up, perhaps we ordinary folks could understand what they're talking about.
Author: We could indeed, because then they'd have to leave out all their best ideas.
Critic: That was supposed to be a joke. Anyway, I think I see the central point. It isn't merely arguing that each particular subject could benefit from using its own special symbols. Tonfoni is proposing that a certain particular set of symbols could help us with texts in general. That could improve all sorts of writing, at only a single initial cost.
Source: http://www.dcs.exeter.ac.uk/~masoud/yazdani/friend/tonfoni/t-book.htm
Back to the Top or return to the Essays Index