Various contentions of the grammatical approach to pattern recognition are critically examined. The most general tenet of this school of thought is that a pattern should be considered as composed of components of known types arranged in a certain fashion. One of the most specific assumptions is that the rules governing the arrangement of components form a “finite” phrase structure grammar. There is a whole gamut of viewpoints between these two poles. Their grammatical theories, however, are usually ungrammatical in some strict sense. We introduce two examples of structure description, one of pictorial patterns with a set of group-theoretical rules and the other a set of rules for linearization of chemical structural formulae. They are both grammatical in the sense that they are based on production rules. But the first is ungrammatical because it is so strongly Gestalt-forming that no local substitution is allowed. The second is ungrammatical because the grammaticalness here cannot define the set of actually existing molecules.