History
SNOBOL
The original SNOBOL effort, retroactively known as SNOBOL1, launched in the fall of 1962 at the Bell Labs Programming Research Studies Department. The effort was a reaction to the frustrations of attempting to use the SCL language for polynomial formula manipulation, symbolic integration and studying Markov chains. SCL, written by the department head Chester Lee, was both slow and had a low-level syntax that resulting in volumes of code for even simple projects. After briefly considering the COMIT language, Ivan Polonsky, Ralph Griswold and David Farber, all members of the six-person department, decided to write their own language to solve these problems.
The first versions were running on the IBM 7090 in early 1963, and by the summer had been built out and was being used across Bell. This led almost immediately to SNOBOL2, which added a number of built-in functions, and the ability to link to external assembly language code. It was released in April 1964 and mostly used within Bell, but also saw some use at Project MAC. The introduction of system functions served mostly to indicate the need for user functions, which was the major feature of SNOBOL3, released in July 1964.
SNOBOL3's introduction corresponded with major changes within the Bell Labs computing department, including the addition of the new GE 645 mainframe which would require a rewrite of SNOBOL. Instead, the team suggested writing a new version that would run on a virtual machine, named SIL for SNOBOL Intermediate Language, allowing it to be easily ported to any sufficiently powerful platform. This proposal was accepted as SNOBOL4 in September 1965. By this time, plans for a significantly improved version of the language emerged in August 1966. Further work on the language continued throughout the rest of the 1960s, notably adding the associative array type in later version, which they referred to as a table.
SL5 leads to Icon
Griswold left Bell Labs to become a professor at the University of Arizona in August 1971. He introduced SNOBOL4 as a research tool at that time. He received grants from the National Science Foundation to continue supporting and evolving SNOBOL.
As a language originally developed in the early 1960s, SNOBOL's syntax bears the marks of other early programming languages like FORTRAN and COBOL. In particular, the language is column-dependant, as many of these languages were entered on punch cards where column layout is natural. Additionally, control structures were almost entirely based on branching around code rather than the use of blocks, which were becoming a must-have feature after the introduction of ALGOL 60. By the time he moved to Arizona, the syntax of SNOBOL4 was hopelessly outdated.
Griswold began the effort of implementing SNOBOL's underlying success/failure concept with traditional flow control structures like if/then. This became SL5, short for "SNOBOL Language 5", but the result was unsatisfying. In 1977, he returned to the language to consider a new version. He abandoned the very powerful function system introduced in SL5 with a simpler concept of suspend/resume and developed a new concept for the natural successor to SNOBOL4 with the following principles;
- SNOBOL4's philosophic and sematic basis
- SL5 syntactic basis
- SL5 features, excluding the generalized procedure mechanism
The new language was initially known as SNOBOL5, but as it was significantly different from SNOBOL in all but the underlying concept, a new name was ultimately desired. After considering "s" as a sort of homage to "C", but this was ultimately abandoned due to the problems with typesetting documents using that name. A series of new names were proposed and abandoned; Irving, bard, and "TL" for "The Language". It was at this time that Xerox PARC began publishing about their work on graphical user interfaces and the term "icon" began to enter the computer lexicon. The decision was made to change the name initially to "icon" before finally choosing "Icon".[a]
Language
Basic syntax
The Icon language is derived from the ALGOL-class of structured programming languages, and thus has syntax similar to C or Pascal. Icon is most similar to Pascal, using := syntax for assignments, the procedure keyword and similar syntax. On the other hand, Icon uses C-style braces for structuring execution groups, and programs start by running a procedure called main.
In many ways Icon also shares features with most scripting languages (as well as SNOBOL and SL5, from which they were taken): variables do not have to be declared, types are cast automatically, and numbers can be converted to strings and back automatically. Another feature common to many scripting languages, but not all, is the lack of a line-ending character; in Icon, lines that do not end with a semicolon get ended by an implied semicolon if it makes sense.
Procedures are the basic building blocks of Icon programs. Although they use Pascal naming, they work more like C functions and can return values; there is no function keyword in Icon.
procedure doSomething(aString)
write(aString)
end
Goal-directed execution
One of the key concepts in SNOBOL was that its functions returned the "success" or "failure" as primitives of the language rather than using magic numbers or other techniques. For instance, a function that returns the position of a substring within another string is a common routine found in most language runtime systems; in JavaScript one might want to find the position of the word "World" within a "Hello, World!" program, which would be done with position = "Hello, World".indexOf("World"), which would return 7. If one instead asks for the position = "Hello, World".indexOf("Goodbye") the code will "fail", as the search term does not appear in the string. In JavaScript, as in most languages, this will be indicated by returning a magic number, in this case -1.[19]
In SNOBOL a failure of this sort returns a special value, fail. SNOBOL's syntax operates directly on the success or failure of the operation, jumping to labelled sections of the code without having to write a separate test. For instance, the following code prints "Hello, world!" five times:[20]
* SNOBOL program to print Hello World
I = 1
LOOP OUTPUT = "Hello, world!"
I = I + 1
LE(I, 5) : S(LOOP)
END
To perform the loop, the less-than-or-equal operator, LE, is called on the index variable I, and if it Succeeds, meaning I is less than 5, it branches to the named label LOOP and continues.[20]
Icon retained the concept of flow control based on success or failure but developed the language further. One change was the replacement of the labelled GOTO-like branching with block-oriented structures in keeping with the structured programming style that was sweeping the computer industry in the late 1960s. The second was to allow "failure" to be passed along the call chain so that entire blocks will succeed or fail as a whole. This is a key concept of the Icon language. Whereas in traditional languages one would have to include code to test the success or failure based on boolean logic and then branch based on the outcome, such tests and branches are inherent to Icon code and do not have to be explicitly written.
For instance, consider this bit of code written in the Java programming language. It calls the function read() to read a character from a (previously opened) file, assigns the result to the variable a, and then writes the value of a to another file. The result is to copy one file to another. read will eventually run out of characters to read from the file, potentially on its very first call, which would leave a in an undetermined state and potentially cause write to cause a null pointer exception. To avoid this, read returns the special value EOF (end-of-file) in this situation, which requires an explicit test to avoid writeing it:
while ((a = read()) != EOF) {
write(a);
}
In contrast, in Icon the read() function returns a line of text or &fail. &fail is not simply an analog of EOF, as it is explicitly understood by the language to mean "stop processing" or "do the fail case" depending on the context. The equivalent code in Icon is:
while a := read() do write(a)
This means, "as long as read does not fail, call write, otherwise stop". There is no need to specify a test against the magic number as in the Java example, this is implicit, and the resulting code is simplified. Because success and failure are passed up through the call chain, one can embed function calls within others and they stop when the nested function call fails. For instance, the code above can be reduced to:
In this version, if the read call fails, the write call fails, and the while stops. Icon's branching and looping constructs are all based on the success or failure of the code inside them, not on an arbitrary boolean test provided by the programmer. if performs the then block if its "test" returns a value, and performs the else block or moves to the next line if it returns &fail. Likewise, while continues calling its block until it receives a fail. Icon refers to this concept as goal-directed execution.
It is important to contrast the concept of success and failure with the concept of an exception; exceptions are unusual situations, not expected outcomes. Fails in Icon are expected outcomes; reaching the end of a file is an expected situation and not an exception. Icon does not have exception handling in the traditional sense, although fail is often used in exception-like situations. For instance, if the file being read does not exist, read fails without a special situation being indicated. In traditional language, these "other conditions" have no natural way of being indicated; additional magic numbers may be used, but more typically exception handling is used to "throw" a value. For instance, to handle a missing file in the Java code, one might see:
try {
while ((a = read()) != EOF) {
write(a);
}
} catch (Exception e) {
// something else went wrong, use this catch to exit the loop
}
This case needs two comparisons: one for EOF and another for all other errors. Since Java does not allow exceptions to be compared as logic elements, as under Icon, the lengthy try/catch syntax must be used instead. Try blocks also impose a performance penalty even if no exception is thrown, a distributed cost that Icon normally avoids.
Icon uses this same goal-directed mechanism to perform traditional boolean tests, although with subtle differences. A simple comparison like if a < b then write("a is smaller than b") does not mean, "if the conditional expression evaluation results in or returns a true value" as they would under most languages; instead, it means something more like, "if the conditional expression, here < operation, succeeds and does not fail". In this case, the < operator succeeds if the comparison is true. The if calls its then clause if the expression succeeds, or the else or the next line if it fails. The result is similar to the traditional if/then seen in other languages, the if performs then if a is less than b. The subtlety is that the same comparison expression can be placed anywhere, for instance:
Another difference is that the < operator returns its second argument if it succeeds, which in this example will result in the value of b being written if it is larger than a, otherwise nothing is written. As this is not a test per se, but an operator that returns a value, they can be strung together allowing things like if a < b < c, a common type of comparison that in most languages must be written as a conjunction of two inequalities like if (a < b) && (b < c).
A key aspect of goal-directed execution is that the program may have to rewind to an earlier state if a procedure fails, a task known as backtracking. For instance, consider code that sets a variable to a starting location and then performs operations that may change the value - this is common in string scanning operations for instance, which will advance a cursor through the string as it scans. If the procedure fails, it is important that any subsequent reads of that variable return the original state, not the state as it was being internally manipulated. For this task, Icon has the reversible assignment operator, <-, and the reversible exchange, <->. For instance, consider some code that is attempting to find a pattern string within a larger string:
{
(i := 10) &
(j := (i < find(pattern, inString)))
}
This code begins by moving i to 10, the starting location for the search. However, if the find fails, the block will fail as a whole, which results in the value of i being left at 10 as an undesirable side effect. Replacing i := 10 with i <- 10 indicates that i should be reset to its previous value if the block fails. This provides an analog of atomicity in the execution.
Generators
Expressions in Icon may return a single value, for instance, 5 > x will evaluate and return x if the value of x is less than 5, or else fail. However, Icon also includes the concept of procedures that do not immediately return success or failure, and instead return new values every time they are called. These are known as generators, and are a key part of the Icon language. Within the parlance of Icon, the evaluation of an expression or function produces a result sequence. A result sequence contains all the possible values that can be generated by the expression or function. When the result sequence is exhausted, the expression or function fails.
Icon allows any procedure to return a single value or multiple values, controlled using the fail, return and suspend keywords. A procedure that lacks any of these keywords returns &fail, which occurs whenever execution runs to the end of a procedure. For instance:
procedure f(x)
if x > 0 then {
return 1
}
end
Calling f(5) will return 1, but calling f(-1) will return &fail. This can lead to non-obvious behavior, for instance, write(f(-1)) will output nothing because f fails and suspends operation of write.
Converting a procedure to be a generator uses the suspend keyword, which means "return this value, and when called again, start execution at this point". In this respect it is something like a combination of the static concept in C and return. For instance:
procedure ItoJ(i, j)
while i <= j do {
suspend i
i +:= 1
}
fail
end
creates a generator that returns a series of numbers starting at i and ending a j, and then returns &fail after that.[b] The suspend i stops execution and returns the value of i without reseting any of the state. When another call is made to the same function, execution picks up at that point with the previous values. In this case, that causes it to perform i +:= 1, loop back to the start of the while block, and then return the next value and suspend again. This continues until i <= j fails, at which point it exits the block and calls fail. This allows iterators to be constructed with ease.
Another type of generator-builder is the alternator, which looks and operates like the boolean or operator. For instance:
if y < (x | 5) then write("y=", y)
This appears to say "if y is smaller than x or 5 then...", but is actually a short-form for a generator that returns values until it falls off the end of the list. The values of the list are "injected" into the operations, in this case, <. So in this example, the system first tests y < x, if x is indeed larger than y it returns the value of x, the test passes, and the value of y is written out in the then clause. However, if x is not larger than y it fails, and the alternator continues, performing y < 5. If that test passes, y is written. If y is smaller than neither x or 5, the alternator runs out of tests and fails, the if fails, and the write is not performed. Thus, the value of y will appear on the console if it is smaller than x or 5, thereby fulfilling the purpose of a boolean or. Functions will not be called unless evaluating their parameters succeeds, so this example can be shortened to:
Internally, the alternator is not simply an or and one can also use it to construct arbitrary lists of values. This can be used to iterate over arbitrary values, like:
every i := (1|3|4|5|10|11|23) do write(i)
As lists of integers are commonly found in many programming contexts, Icon also includes the to keyword to construct ad hoc integer generators:
every k := i to j do write(k)
which can be shortened:
Icon is not strongly typed, so the alternator lists can contain different types of items:
every i := (1 | "hello" | x < 5) do write(i)
This writes 1, "hello" and maybe 5 depending on the value of x.
Likewise the conjunction operator, &, is used in a fashion similar to a boolean and operator:
every x := ItoJ(0,10) & x % 2 == 0 do write(x)
This code calls ItoJ and returns an initial value of 0 which is assigned to x. It then performs the right-hand side of the conjunction, and since x % 2 does equal 0, it writes out the value. It then calls the ItoJ generator again which assigns 1 to x, which fails the right-hand-side and prints nothing. The result is a list of every even integer from 0 to 10.
The concept of generators is particularly useful and powerful when used with string operations, and is a major underlying basis for Icon's overall design. Consider the indexOf operation found in many languages; this function looks for one string within another and returns an index of its location, or a magic number if it is not found. For instance:
s = "All the world's a stage. And all the men and women merely players";
i = indexOf("the", s);
write(i);
This will scan the string s, find the first occurrence of "the", and return that index, in this case 4. The string, however, contains two instances of the string "the", so to return the second example an alternate syntax is used:
j = indexOf("the", s, i+1);
write(j);
This tells it to scan starting at location 5, so it will not match the first instance we found previously. However, there may not be a second instance of "the" -there may not be a first one either- so the return value from indexOf has to be checked against the magic number -1 which is used to indicate no matches. A complete routine that prints out the location of every instance is:
s = "All the world's a stage. And all the men and women merely players";
i = indexOf("the", s);
while i != -1 {
write(i);
i = indexOf("the", s, i+1);
}
In Icon, the equivalent find is a generator, so the same results can be created with a single line:
s := "All the world's a stage. And all the men and women merely players"
every write(find("the", s))
Of course there are times where one does want to find a string after some point in input, for instance, if scanning a text file that contains a line number in the first four columns, a space, and then a line of text. Goal-directed execution can be used to skip over the line numbers:
every write(5 < find("the", s))
The position will only be returned if "the" appears after position 5; the comparison will fail otherwise, pass the fail to write, and the write will not occur.
The every operator is similar to while, looping through every item returned by a generator and exiting on failure:
every k := i to j do
write(someFunction(k))
There is a key difference between every and while; while re-evaluates the first result until it fails, whereas every fetches the next value from a generator. every actually injects values into the function in a fashion similar to blocks under Smalltalk. For instance, the above loop can be re-written this way:
every write(someFunction(i to j))
In this case, the values from i to j will be injected into someFunction and (potentially) write multiple lines of output.
Collections
Icon includes several collection types including lists that can also be used as stacks and queues, tables (also known as maps or dictionaries in other languages), sets and others. Icon refers to these as structures. Collections are inherent generators and can be easily called using the bang syntax. For instance:
lines := [] # create an empty list
while line := read() do { # loop reading lines from standard input
push(lines, line) # use stack-like syntax to push the line on the list
}
while line := pop(lines) do { # loop while lines can be popped off the list
write(line) # write the line out
}
Using the fail propagation as seen in earlier examples, we can combine the tests and the loops:
lines := [] # create an empty list
while push(lines, read()) # push until empty
while write(pop(lines)) # write until empty
Because the list collection is a generator, this can be further simplified with the bang syntax:
lines := []
every push(lines, !&input)
every write(!lines)
In this case, the bang in write causes Icon to return a line of text one by one from the array and finally fail at the end. &input is a generator-based analog of read that reads a line from standard input, so !&input continues reading lines until the file ends.
As Icon is typeless, lists can contain any different types of values:
aCat := ["muffins", "tabby", 2002, 8]
The items can included other structures. To build larger lists, Icon includes the list generator; i := list(10, "word") generates a list containing 10 copies of "word". Like arrays in other languages, Icon allows items to be looked up by position, e.g., weight := aCat[4]. Array slicing is included, allowing new lists to be created out of the elements of other lists, for instance, aCat := Cats[2:4] produces a new list called aCat that contains "tabby" and 2002.
Tables are essentially lists with arbitrary index keys rather than integers:
symbols := table(0)
symbols["there"] := 1
symbols["here"] := 2
This code creates a table that will use zero as the default value of any unknown key. It then adds two items into the table, with the keys "there" and "here", and values 1 and 2.
Sets are also similar to lists but contain only a single member of any given value. Icon includes the ++ to produce the union of two sets, ** the intersection, and -- the difference. Icon includes a number of pre-defined "Cset"s, a set containing various characters. There are four standard Csets in Icon, &ucase, &lcase, &letters, and &digits. New Csets can be made by enclosing a string in single quotes, for instance, vowel := 'aeiou'.
Strings
In Icon, strings are lists of characters. As a list, they are generators and can thus be iterated over using the bang syntax:
Will print out each character of the string on a separate line.
Substrings can be extracted from a string by using a range specification within brackets. A range specification can return a point to a single character, or a slice of the string. Strings can be indexed from either the right or the left. Positions within a string are defined to be between the characters 1A2B3C4 and can be specified from the right −3A−2B−1C0
For example,
"Wikipedia"[1] ==> "W"
"Wikipedia"[3] ==> "k"
"Wikipedia"[0] ==> "a"
"Wikipedia"[1:3] ==> "Wi"
"Wikipedia"[-2:0] ==> "ia"
"Wikipedia"[2+:3] ==> "iki"
Where the last example shows using a length instead of an ending position
The subscripting specification can be used as a lvalue within an expression. This can be used to insert strings into another string or delete parts of a string. For example:
s := "abc"
s[2] := "123"
s now has a value of "a123c"
s := "abcdefg"
s[3:5] := "ABCD"
s now has a value of "abABCDefg"
s := "abcdefg"
s[3:5] := ""
s now has a value of "abefg"
String scanning
A further simplification for handling strings is the scanning system, invoked with ?, which calls functions on a string:
Icon refers to the left-hand-side of the ? as the subject, and passes it into string functions. Recall the find takes two parameters, the search text as parameter one and the string to search in parameter two. Using ? the second parameter is implicit and does not have to be specified by the programmer. In the common cases when multiple functions are being called on a single string in sequence, this style can significantly reduce the length of the resulting code and improve clarity. Icon function signatures identify the subject parameter in their definitions so the parameter can be hoisted in this fashion.
The ? is not simply a form of syntactic sugar, it also sets up a "string scanning environment" for any following string operations. This is based on two internal variables, &subject and &pos; &subject is simply a pointer to the original string, while &pos is the current position within it, or cursor. Icon's various string manipulation procedures use these two variables so they do not have to be explicitly supplied by the programmer. For example:
s := "this is a string"
s ? write("subject=[",&subject,"], pos=[",&pos,"]")
would produce:
subject=[this is a string], pos=[1]
Built-in and user-defined functions can be used to move around within the string being scanned. All of the built-in functions will default to &subject and &pos to allow the scanning syntax to be used. The following code will write all blank-delimited "words" in a string:
s := "this is a string"
s ? { # Establish string scanning environment
while not pos(0) do { # Test for end of string
tab(many(' ')) # Skip past any blanks
word := tab(upto(' ') | 0) # the next word is up to the next blank -or- the end of the line
write(word) # write the word
}
}
There are a number of new functions introduced in this example. pos returns the current value of &pos. It may not be immediately obvious why one would need this function and not simply use the value of &pos directly; the reason is that &pos is a variable and thus cannot take on the value &fail, which the procedure pos can. Thus pos provides a lightweight wrapper on &pos that allows Icon's goal-directed flow control to be easily used without having to provide hand-written boolean tests against &pos. In this case, the test is "is &pos zero", which, in the odd numbering of Icon's string locations, is the end of the line. If it is not zero, pos returns &fail, which is inverted with the not and the loop continues.
many finds one or more examples of the provided Cset parameter starting at the current &pos. In this case, it is looking for space characters, so the result of this function is the location of the first non-space character after &pos. tab moves &pos to that location, again with a potential &fail in case, for instance, many falls off the end of the string. upto is essentially the reverse of many; it returns the location immediately prior to its provided Cset, which the example then sets the &pos to with another tab. Alternation is used to also stop at the end of a line.
This example can be made more robust through the use of a more appropriate "word breaking" Cset which might include periods, commas and other punctuation, as well as other whitespace characters like tab and non-breaking spaces. That Cset can then be used in many and upto.
A more complex example demonstrates the integration of generators and string scanning within the language.
procedure main()
s := "Mon Dec 8"
s ? write(Mdate() | "not a valid date")
end
# Define a matching function that returns
# a string that matches a day month dayofmonth
procedure Mdate()
# Define some initial values
static dates
static days
initial {
days := ["Mon","Tue","Wed","Thr","Fri","Sat","Sun"]
months := ["Jan","Feb","Mar","Apr","May","Jun",
"Jul","Aug","Sep","Oct","Nov","Dec"]
}
every suspend (retval <- tab(match(!days)) || # Match a day
=" " || # Followed by a blank
tab(match(!months)) || # Followed by the month
=" " || # Followed by a blank
matchdigits(2) # Followed by at least 2 digits
) &
(=" " | pos(0) ) & # Either a blank or the end of the string
retval # And finally return the string
end
# Matching function that returns a string of n digits
procedure matchdigits(n)
suspend (v := tab(many(&digits)) & *v <= n) & v
end
