Visualizing Movie Data | VMD_01 | Mapping Movie Data

In 2008 we began reviewing movies. In the beginning we were doing that using points that lay between 0 and 10. At a later date we used a more narrative way. That led to detailed reviews of movies that we posted on Facebook. But over time was that too much work and it cost too much time so I decided to introduce a more accurate way of reviewing the movies. I started on 5 January 2015 to use 13 categories for each movie. Storyline, originality, cinematography, involvement, sound, editing, educational, title design, acting, interesting, unusual, exciting and superior. Every category earns a score between 0 and 10. A simple Processing program add’s all points together. And the result of this addition is divided by the number of categories. Then you get an average point for one movie. The aim of Visualizing Movie Data is to give us more insight into the choices we make in evaluating a film. Ultimately, this should tell more about ourselves than about the films. If that’s true, that would be a positive spin-off. Through Visualizing Movie Data I try different ways of visualizing data collected by us. And I get help from Ben Fry’s book ‘Visualizing Data’ published by O’Reilly Media Inc. Contrary to the Generative Design Variations project Visualizing Movie Data is not about making as many as possible variations. Now it is the intention that the movie data is used as functional and as basic as possible. I will skip any form of decoration. In this chapter I try to use a very simple and basic way of reading, displaying and interacting with a number of small data sets. These datasets consist partly of the data we found in our movie reviews.

I started looking for a world map. Eventually I found a world map that uses the Mercator projection. Mercator projection is a conformal cylinder projection with large surface deformations at higher latitudes. In this projection Europe is slightly larger which is an advantage because a lot is going to happen in this ‘small’ part of the world. Perhaps it is ultimately necessary to use a separate map of Europe as an insert. But I don’t know that at this moment. I decided to omit all color because I want to use the color for the markers of the countries. At this stage I only display a world map.

This is a first version in which the program reads coördinates from a text file to place red dots. I wanted to know if that worked. And If that works then it should also work with exact coördinates.

Now I need to gather a list of all 46 countries that have produced films. I happened to find a two-letter code list of countries through the International Organization for Standardization (ISO 3166 Country Codes). These abbreviations I can use to display in a later stage. The overall list is much too extensive so I have to select only those countries that have made films that we actually have seen from the beginning of 2015. If you look at this visualization there are a few things that stand out. In Europe it is very busy. Some countries are completely covered by a red dot. And some dots overlap other spots. Actually, it’s a mess. On all other continents there are only a few spots. It looks empty. Further, the overall color of the image looks too dark. I also have made an outline version of the dots. But this does not solve the problem. In Europe the small countries are slightly more visible but it is a minimal improvement and no real solution.

This version I’m going to make an insert for Europe. At least so I thought at that time. After I had made a rough sketch it actually delivered more problems than a solution. In the first place Europe is out of proportion when you compare it with the rest of the world map. Which is the case anyway at each world map. An exception is the projection of Goode. That has is an equal-area map projection. And in the second place an insert covers always a portion of the world map. Making that to be moved again. And actually you create five continents, which are all out of proportion. The question is whether that’s good. So it seemed best to temporarily leave everything as it is. And I solve problems when they are relevant. The dots are slightly reduced and everything becomes pretty clear. I have also added a title plus additional information which makes it even more complete.

A few years ago I have worked on patterns and photography in Processing. I could apply one of those patterns on the world map. Since a realistic world map is not possible anyway, it is just as good to make an abstraction of the world map. The question is now: is abstraction decoration? The world map is now completed with dots with a diameter of 4 pixels. The locations are displayed (as much as possible) in the middle of the country. Although the middle in some countries is hard to find. Where is the middle of the USA with its territories and various possessions?

Those dots are of course meaningless. They only give the central locations of countries where movies are made. It would for instance be more helpful when you could see where the most films are made. This would be able when you could vary the size of the dot. Large dot is a lot of movies. Small dots are a few films. This is a version that uses random generated numbers.

This version uses our movie data. The size of the dot determines the number of films from that country. It is immediately clear that most films come from France, USA, UK and Germany. This of course says nothing about the quality of the movie. It only shows information about the quantity. And it is only partly the truth as we will see later. And how will this visualization look like at the end of 2015 or 2016 as we have seen more movies. It was a surprise that France plays such a leading role in the field of film production. But it also raises immediately questions. Maybe we’re being manipulated? Might there be another variable which makes that France appears so high in the movie production?

It is also possible to interpolate between two colors, and make all dots the same size. I go for the low numbers in red and green for the high numbers. But I find that this version does not really show the smaller differences very well. In fact, there are only two green dots and two interpolations between green and red tending towards brown. The rest is nuances in red. And what does green-ish or reddish-brown mean then?

All countries of which we have seen five or more films are green. The largest dot on the world map indicates that we have seen much more than five movies. A small dot indicates that we have seen five movies. All countries of which we have seen less than five movies are red. A larger red dot indicates that we have seen almost 5 movies. A small dot indicates that we may just have seen only one movie. In short, very complicated and not very clear. And you really need some textual information here.

Of course you do want to be more accurate in displaying this data. So I’ve changed the program in a way that the abbreviated name of the country is displayed when you get to the dot of the country with the cursor. This goes not flawless though. Sometimes the name of the country disappears under a dot. At a certain point when the countries are small and close together both country names are being activated. I made a version of the Futura Medium 12 and with Futura Bold 12. I think that the Futura Bold version is better suited because it is more readable.

I have optimized the mouse interaction. Now, there are never two countries selected at the same time and it is also true that the name of a country is always drawn last. So it can never be overwritten by a dot.

I have replaced the floating numbers with integers. And I replaced all the colors by green. I think this is a reasonable version. You can easily see from which country most films come from. Ad the for the exact quantities you get feedback from the cursor.

I’m going a little deeper into the movie productions made in France, United Kingdom, Germany and the USA. I begin with France. That data does not look very spectacular. Apparently, most of the movies are filmed in Paris. In addition there are a few film locations in the south and center of France. Furthermore it seems that of the 47 French films that we have seen only 20 are filmed on a location in France. The other 27 are made through partnerships with other countries. And the film locations are all situated outside of France.

There is a problem with the data. When I run the program I get an ArrayIndexOutOfBoundsException: 8 error. I tried to find out what causes this error. It appears when you read data from text files that are of different lengths you get an error. That seems logical because in that way the arrays can not all be filled just the same way. If you have an array with 10 lines and another with 8 lines you get this error. And that was the case when I went to adjust the French data files for the United Kingdom data files. The 8 in the error message is the number of lines that were set aside for the array.

It looks empty in Germany. But I think it all will be fine in the long-term. Also, I think I’ve done something wrong. Some movies, of course, have several film locations. If I can trace those it is guaranteed that the image is getting more interesting.

Also this US version is not really interesting. But something else is happening. If the first two characters in my text file are not unique (and thus are duplicates in the list) the positioning of the dots and text goes wrong. And because the first two characters have no further function (but apparently have influence) it might be better to keep this form of abbreviations: AA, AB, AC, AD, AE, etc., and after AZ continuing with BA, BB, BC, BD.

I have now found all film locations for all American films we have seen since January 2015. I did a much too  superficially search so I found only 18 locations. Now I have 218 locations available. A number of them will not be used because the film locations are outside the USA. And there are several movies that play at the same location. So the final list will be shorter than 218 but longer than 18. Now I need to make a list and avoid duplication. All the movies filmed in Los Angeles are to be summed up to a total. All films in New York. All films of Detroit. And this goes for all the other cities and towns in the USA. I ended up with a final list of 87 cities and villages. Los Angeles has reached the top 61 productions. Which is actually not very surprising.

I now go one level deeper. I started on a global scale. Then countries scale. And I’m now going to work on an urban scale. After some research it seemed suitable to me to use the film locations of the series ‘Breaking Bad’. I display the abstract version of the map of Albuquerque in the background. And I used a reduced version of the Breaking Bad wordmark. But that doesn’t work at all. I also find the amount of film locations insufficient.

After some more research I could trace a lot more ‘Breaking Bad’ film locations. I changed the title and background colors. But I think it’s really ugly. When something is designed simple and basically it doesn’t have to look ugly. So I have to work on that.

This is the state in which I want to finish this exercise for the time being. All locations can now be read and the amount of scenes are shown after the location name. There is a title and a subtile. You can immediately see that Walter White’s House is the film location which is most used (with 80 scenes). Then Jesse Pinkman’s House follows (with 42 scenes). Hank and Marie’s House (with 29 scenes). The DEA offices (with 27 scenes). The Car Wash (with 22 scenes), Jesse Pinkman & Jane’s House (with 21), Gus’s Laundry Service (with 20) and Los Pollos Hermanos (with 17). Too bad I can not show this in a JavaScript version because that is unfortunately not working.

A rough conclusion: Data visualization is much harder than dreaming up nice effects like I did in the previous Generative Design Variations project. In data visualization you should limit yourself in order not to come up with lots nonfunctional decoration. A lot of time must be invested in research in the beginning, during and sometimes afterwards the design phase. And it is an iterative process. There are always improvements possible after the improvement. It also takes more time than a general design job. You must be very precise and constantly looking for better data and better interpretation and visualization of the data. Data visualization has a high level of detective work.

Generative Design Variations M.6.6 Fish-eye view

This is the last exercise of the Generative Design Variations project. Technically I have modified 1219 programs. The programs generated 5957 images. And I have written 83 articles about this project which lasted from November 10, 2013 until  August 21, 2015.

Here is the summary from the Generative Design book: ‘The more nodes added to the graph, the more the network expands in all directions. One can zoom out in order to gain an overview, but the nodes, and especially their inscriptions, lose their legibility. To resolve this discrepancy, the area in which the graph is displayed can be distorted with a fish-eye projection. This functions like a wide-angle lens with a 180º angle. The elements in the middle of the display are still depicted at their original size, but te farther the elements are from the center, the smaller they are drawn–although they remain visible in the display. This kind of distortion can help maintain perspective without distorting information, especially with representations like force-directed layouts, in which the linking structure of the nodes is more important than their exact position. Projection always means taking the original coördinates of the points to be displayed (in this case the coördinates of the nodes) and calculating new coördinates. Only the latter coördinates are used to draw the object in the display.’ So far this summary. And here you can find the the original program.

JavaScript does not support the Generative Design library. Or vice versa. I think the class Processing XML data is also not supported. But I am not sure about that. Furthermore, there are some obscure things imported during the initialization phase:*,, java.util.regex.*, java.util.Calendar, java.util.Iterator and java.util.Map. Because this program is a tool and because I modified it not much I thought it was not necessary to put it online. But I have created a Flickr album where all the images are that I have made during this exercise.

This is still the same tool I used in the previous assignments. I only have chosen to use a little more functionality. And as I said in the previous exercise it is now my intention to use all functionality of the tool to create images. I repeat once again the basic elements of the tool (for myself):
– any element (or node) reflects an article from Wikipedia pages
– the themes are science, nature and society, art and culture
– in the original setup of the tool science is blue, nature and society is yellow, art and culture is purple
– the size of the element is decisive for the length of the article
– there should be an indication in the node that represents the amount of links
– another element should show that there are additional items.

I have made a few screen dumps from the arrangement as it was in the earlier exercise (M.6.5). So the colors I have used in this previous set-up are changed and do not correspond to the above list. Any node reflects a Wikipedia article. And the keywords that I would like to use are coming from people who are working in the science, nature, society, art and cultural world. I start with: Louis B Mayer (1885-1957), US film producer.

Keyword: Maria Callas (1923-1977), Operatic soprano. In this graphic are three theme colors. But I want to modify those because I think there are more possibilities. Science is now red, nature and society are yellow and  art and culture is blue. And looking a bit deeper into this linking mechanism it seems that Maria Callas was little concerned with science. That is why there is only a bit red present in the image. The majority is art, which is blue-ish. Of course it depends on how many nodes you click. But there also are interesting relationships noticeable. For example, what has Maria Callas to do with L-Dopa? I continue to change colors during the following exercises.

Keyword: Ralph Waldo Emerson (1803-1882), US writer. Because the colors are interpolated it seems better to me to choose three very different colors. I have made science red, nature and society white, art and culture is blue. The disadvantage of white is that it brings back all bright colors to a shade or a tint of that color. So I replaced white with magenta. Probably this is not a good choice but I think yellow is still good. Although I think it’s a very ugly color combination. I experiment now by exaggerating the shapes. Which also leads to terrible results if you ask me.

Keyword: Alma Mahler Werfel (1879-1964, Wife of composer Gustav Mahler, then architect Walter Gropius, and finally writer Franz Werfel. There are now three bars one above the other. The white bar is there only if there are additional items. The middle bar represents the themes and the lower bar is there if there are any links.

Keyword: Arthur Wellesley 1st Duke of Wellington (1769-1852), British general and statesman. What happens if I drop all these circles and just start with texts and lines. Maybe I should also cut the arrow points for a moment but it could well be that they eventually disappear forever. Finally, the main thing that I am interested in is the article’s name. So I’m going to bring this chart all the way back to its essence.

Keyword: George Sand (1804-1876), French novelist and memoirist. I have not much used the feature that lets you specify colored lines. That gives a completely different picture.

Keyword: Ludwig Mies van der Rohe (1886-1969), German-American architect. Color was added to the central point that indicates that there are back links. I think the point maybe slightly larger. Actually, I make the same mistake again. The emphasis is now on the nodes. While I would like the emphasis to be on the texts.

Keyword: Eva Duarte de Perón (1919-1952), Actress and First Lady of Argentina. I have associated the colors with the text. So now you see directly in which group the texts fall. Over time the colored texts disappear but the context of the categories is taken over by the colored dots in the nodes.

Keyword: Groucho Marx (1895-1977), Comedian. I tried to find out how many links are possible. But only so that it all remains manageable. I also think that those black circles in the background do a great job. I leave them in.

And let’s finally spend the last keyword to the woman to whom we owe all this programming: Ada, Countess of  Lovelace (1815-1852), English mathematician, writer and pioneer of computing. And the Wikipedia link is not working. What is wrong? Nothing! Just kidding!

Generative Design Variations M.6.5 Semantic text analysis

Here is a copy of the summary from the Generative Design book: ‘Now the nodes can tell us how significant a Wikipedia article is. Other than the title, there is still no way to infer the content of the article. It would be useful when, for example, the color of the node reflected its thematic affiliation–if the article is about science, art or culture or of it concerns geographic or political subjects. Unfortunately, Wikipedia does not supply this information. It is possible, however, to implement a simple semantic text analysis. This means that keywords in the text are defined and counted. The more often the keywords of a particular subject appear and the less frequently others do, the more likely it is that the article is about that one subject. If colors are assigned to subjects, then the frequency of the keywords can be used to interpolate between these colors.’ So far this summary. Here is the original program.

JavaScript does not support the Generative Design library. Or vice versa. I think that the Processing XML data is also not supported. Furthermore, there are some (for me obscure) things imported when initialized:, java.util.regex, java.util.Calendar, java.util.Iterator and java.util.Map. This program is a tool and that is why I thought it was not necessary to put my versions it online. I did not change a lot. Most changes are about the visual look of what the tool generates. I have created a Flickr album where all the images are that I have made during this exercise.

This exercise is about the functional use of color. Furthermore, I’m going to try if I can do something about the readability of the text. During the earlier exercise (M.6.4) I had chosen to take design related keywords. Let me choose this time for computer-related words. I begin with the keyword: Computer. Furthermore I switched on colorize nodes. And than some strange things happen. I start the program and it does not work. Only after 3 restarts, it works. I try it again. Now the program works only after starting twice. That’s one thing. Another thing is that I have reduced the some nodes (resultCount) from 50 to 5. The first time I get the following nodes from the keyword ‘Computer’: Kermit (protocol), Computer Monitor, EGB, fighter aircraft and Alan Turing. The second time I get the following nodes: Digital object identifier, Interactive fiction, Floppy disk, first-person shooter, Lisp (programming language) and Digital object identifier. The third time I run the program: Computer data storage, Mac OS, BUNCH, BASIC, and Nano Engineering. So I keep getting different links (nodes) with the same keyword. I’m not sure if I like that a lot. Concerning color, we have three groups: Science is blue. Geography and politics are yellow. Culture and art is purple. The colors are fine. I’ve just made them a bit brighter. And it is still true that I have to start the program at least three times before it works (or runs).

Keyword: Software. I have to think of something that I can improve the readability of the text. I find those gray texts are not (or hardly) legible. So I removed the black rectangle beneath all the texts. Just like I did in the previous exercise. With regard to the size of the font you simply turn off auto zoom. And then it reads all just fine. With auto zoom turned on text will automatically enlarge or reduce. On a certain moment (very large or ver small) the texts are unreadable. To increase the size for all texts (to allow them always be legible) is tricky because on a certain moment you cannot see the graphics anymore.

Keyword: Hardware. I want to see if I can get rid of those circles. I do a search in the program itself and actually all circles are generated in the Wikipedia Node class. I first made a version in which I replaced a portion of the circles by squares.

Keyword: Malware. The color scheme of science is now white, geography and politics are now red. And culture and art is now blue. All shapes now consist solely of squares. Changed the colors again. Science is now blue, geography and politics are now purple and culture and art is now red.

Keyword: Algorithm. I have changed the squares into rectangles. That creates a less chaotic image than the previous images. But if that is any better?

Keyword Boolean. Maybe it’s good to go back to the original version with circles. And maybe those circles do not have to sit on each other but close to each other. Or maybe side by side. This also means that some colors will flow into the color of their neighbor. And I think an outline version also could be an option. But for now I only will give the center dot (for back links) a white outline. There is still one problem with the color of some texts. Some stay gray. Others remain white. I also think that the node’s cricles should be a little closer to each other.

Keyword: Programming language. I think it is time to sort out a few things. I feel that I have strayed a bit. In the original program, each circle reflects a Wikipedia article. The arrows between the articles show whether an article is linked to another article. Then there are three themes. Blue is science, nature and society is yellow and purple is art and culture. The size of the circle indicates the length of the article. And the thickness of the outer ring represents the number of links in the article. The dot in the center indicates if there are multiple items. So I think I should go back to these principles. Maybe I can make another setup.

Keyword: Virtual machine. I have to change the names of some variables in the program. As ‘s’ and ‘b’ and ‘d’ tell me so little about their functionality. I now have a some circles commented out. The circles (who are on), I replaced by rectangles. Eventually I’ve replaced the last circle to a rectangle. And now all the information is translated from circle to rectangle. While this does not give me an image which is wrong it is not entirely true. I do have the impression that it is an improvement on M_6_5_01_GDV_04. But I really want to try something else.

Keyword: Processor. Actually, I want to get rid of the circles because it does not give a proper interpretation of the data. Why? Then I suggest to buy Alberto Cairo’s book: ‘The functional art’. He explains the circle problem better than I can. At first glance, this seems more like a London Underground Map. What I want to try is not to make a square but a horizontal line. I now have two horizontal rectangles. The lower rectangle has a color which  is a percentage of the upper rectangle color. If there is no bottom rectangle, then there is no link. The quantity of links I cannot measure now. And the texts have to be moved out-of-the-way.

Keyword: Microprocessor. I was not able yet to figure out how I could make the visual translation to the quantity of links in the lower rectangle. But you can clearly see if there are any links. So this exercise was not 100 percent successful. But perhaps 75 percent. Let’s see if I succeed to 100% in the next exercise. I’ll also try to bring more structure to the layout of the nodes (as I’ve tried in M_6_4_01_GDV_06). That has to be better in the next assignment.

Generative Design Variations M.6.4.1 Visualizing proportions

Here is the summary of the Generative Design book: ‘Until now the nodes have all looked the same, although they represent completely different Wikipedia articles. For instance, it is not apparent if a represented article is long or short, how many left in it refer to other articles, or if the article itself is refered to. It would make sense to draw the larger nodes when the articles are longer. It must be noted that the quantitative information is recognized by the area of the graphic element and not by its radius. A ring is drawn around the node with a width indicating how many links have yet to be displayed with the arrows originating from the node.’ So much for the summary. And here you can find the original program.

JavaScript does not support the Generative Design library. And I am not sure but I think the Processing XML data class is also not supported. Furthermore, there are some (for me) obscure things imported in this tool:, and some java utilities. And because this program is a tool I did not find it necessary to put the programs online. I have created a Flickr album where all the images can be found that I have made during this exercise.

Extremely annoying to say but to be frank this tool does not work. It displays the keyword Design. And you can add a new node. But that’s all. I started the program several times. That worked sometimes during using the previous program (M.6.3.1). After three times the program than suddenly started. But that does not work now with this program. I have figured out three different ways I can still use this tool. The first way is to import the changes I made in M.6.3.1 (the previous program) and look what happens. The second way is to build on the previous program (M.6.3.1). And then forget about the tool. And the third way is to try to make my version of the tool. That program would have absolutely nothing to do with what is described in the Generative Design book. But that would obviously be sad. Since I have been able to use all the programs so far. I first replaced in the WikipediaNode class all http:// links by https://. And then I was surprised that the program worked normal. I get no more errors in the console. So the problem seems to be solved easily. I reported that on the Generative Design website. Ah … and this could also be one of the causes. I work with a Wacom tablet and a pen. When I start the program with the pen it sometimes does not work. When I start the program with the mouse on the Wacom tablet then it starts most of the times. Sounds very vague. But it is what it is.

As usual I started to type all the book-comments into the code. I also downloaded the complete font family MISO. MISO is an architectural lettering font completed in 2006 by Mârten Nettelbladt. I wanted to test whether any of these fonts (or a combination of them) is also suitable for use in this program. But unfortunately this did not lead to better results. I did not choose for a different keyword. That will happen in the following exercises. The keyword for this exercise is ‘Design’. I am now going to set the first variables. Auto Zoom is true. Invert background is true. I do not know what the variable resultCount represents. So I’m going to increase it from 10 to 50. Ah… resultCount is the number of nodes with which the program starts. I put this back to 1 for now. Then it is much easier to track where all nodes link to. I think a resultCount of 500 is the limit for this program. That number is also as stated in the API lines of the WikipediaNode class. There also occurs a strange side effect in the graphics of this program. The design node begins to vibrate violently when I use high numbers of nodes. I have no idea why. I also removed all color information from the image. Bringing color in the graphics will be the next assignment.

Let’s use design related keywords for these exercises. I will use the keyword ‘Designer’ for this exercise. That’s a tiny difference from the previous keyword: Design. But there is a big difference in the end-result when you put the spring length at 10 or 500. I can influence that amount via the GUI. So these differences are even getting bigger. I reduced the minimum setting. And the doubled maximum setting in the GUI. The disadvantage is that the texts in many nodes can no longer be read. So you may have to zoom in. Actually, this typographically not very wise. Perhaps the texts had to decrease more slowly when creating more nodes. Or switch off automatic zoom.

Keyword: Industrial Designer. I have put the spring stiffness very low. This way you can make beautiful organic structures. If you want to go that way. The tool can also create unexpected node clusters.

Keyword: Design Research. What happens if I double the slider to for the spring stiffness again? Then everything becomes even more interesting. Only the readability goes steps backwards. In this case, I do not care but my clients will appreciate it less.

Keyword: Graphic design. Spring strength and spring stiffness are at their maximum. Spring damping is on its minimum. This results in very fancy circles which are formed by the nodes. As you change the settings the circles are getting more organic again.

Keyword: Industrial design. I Increased the node radius to its maximum. Once to its minimum and one time half of the slider. I also increased the spring length. At a certain moment all nodes pile themselves up. Therefore it seems that there are fewer nodes present. But by playing with the settings the nodes are getting visible again. M_6_4_01_GDV_06

Keyword: Designer drug. Brought every setting to its maximum. But wait … that’s a reassuring warning in my console: Error during asyncHTMLLoad – but no problem.

Keyword: Design Patent. Brought the line weight to an extreme setting. This results graphically in nice images. But I really should do something about the size of the texts. I’m going to try to solve that problem in the next assignment.

Keyword: Software design. Just a variation in which I made the line thickness excessively thick. It is less refined but not really wrong. Plus some screen dumps of the moments that I’ve zoomed in really deep on a constellation of nodes and springs.

Keyword: User interface design. Actually, the gray text on black rectangles is not readable. Perhaps there is also room for improvement for that issue in the next assignment.

Generative Design Variations M.6.3 Force-directed layout with data

This is the summary from the Generative Design book ‘The plan is now to begin with a Wikipedia article and, each time it is clicked, reveal a specific number of links, thereby enabling the linking structure to be gradually expanded. In order to maintain legibility in the increasingly complex program, it is useful to summarize a part of the functionality with classes. The WikipediaNode class expands the node class so that It can load the data as independently as possible. These, and the linked springs, are administered in the class WikipediaGraph.’ So far this summary. Here is the original program.

JavaScript does not support the Generative Design library and I think it does not support Processing data XML class. But I am not sure about that. Further there are some (for me) obscure things imported like:, and a few java utilities. That’s why I thought it was not necessary to put the programs online. I have created a Flickr album where you can find all the images that I have made during this exercise.

The first thing I tried is to take a different keyword for each exercise and not the keyword ‘Superegg’. This time I choose the name ‘Steve Jobs’. And that does not work. Except when I bring back the pllimits in Wikipedia node from 500 to 100. The program also keeps on displaying [Fatal Error]: 1: 1: Premature end of file in the console. But it continues to work. So I do not worry too much about me. Commented out the frame.setResizable (true). I’m not going to the increase or decrease the display window. I keep it always at 800 x 800 pixels. I made the background black. Then it seems that all rectangles below the text there are displayed. That does visually not seem right to me. Which function is responsible for displaying those rectangles? Ah… some of the rectangles dissolve in the background over time. The backgrounds are black now but it would be better if they were completely transparent. I find it unnecessary to display them. Another thing is that the nodes are getting now much more attention than the texts. While the text for me is much more important. Hmmm … there are many more things I can not fix at this time. That is something to be done later.

There is a problem with the color scheme. Two color modes (RGB and HSB) are mixed. Why? And who uses what? Let me first start filling in another keyword. I choose ‘Processing (programming language)’. I am trying to figuring out why I have two different shades of red. Ok… everything is changed and the place to do that can be found in the draw label block in the Wikipedia Node class. All top-nodes are now red. The sub-nodes are blue. Not everything is quite clear to me. I have extended the display time of the sub-nodes. I think they’re too short on the screen.

Keyword: Data Visualization. It seems better to make smaller nodes and fill them with white. Probably it is also better to use a larger point size for the top-nodes. The point size of the sub-notes is fine.

Keyword: Color. When you increase the point size 10 times and when zooming in the point size of the top-nodes is increased again. So I guess I that the point size should stay lower than 1.0. Zooming in the keyword related top-nodes increases their point size. And that’s actually a good thing. Because you eventually want to know where the top-nodes are located. But maybe this is not such a good idea after all. This will become clear later. M_6_3_01_GDV_04

Keyword: Interpolation. I would actually also be able to zoom in and out much deeper. And maybe the length of the spring needs to be shortened. So just for fun, I have increased the node radius to 10. That means I get very short springs. Then I clicked on a large number of links. Which gives me total chaos.

Keyword: Complexity. I have no idea why the nodes are drawn in 3 steps. So I started with commenting out the last dot. This results in one circle less in the node. The second block comment out displays no node at all. I check the code in the original program. Ok. The first circle is drawn to the first node. But apparently I do not see that. The second circle is a white ring between the center circle and the link circle. And the last circle is the point at the center of the node. I exaggerated the second circle in size with a very low alpha channel.

Keyword: Population genetics. I changed the color blue. The texts are more readable now. Perhaps I should give the larger circle a bit more transparency. Finally opted for a line version. But that made the image very unclear and therefore less legible. So I go back to the earlier version. Brought the nodeRadius back to its original setting.
Keyword: Greek history. In one way or another, the program does not always start. Sometimes it does nothing when you let it run. Sometimes it works. I have no idea why. The readability of the blue text is no longer optimal when they are on a light gray rectangles. I could replace the rectangles by horizontal or vertical lines. I now uncomment the original white rectangles. But I think the overall picture did not improve. I have applied a gradient in the background. But I think that those rectangles behind the text are not an improvement.

Keyword: Typography. Time to add some sophistication. I’m going to remove the areas behind the text. And I think that the gradient must begin darker at the bottom otherwise the texts are no longer legible.

Keyword: Dada. A few more optimizations. But I have not changed a lot when you compare it with the earlier version. The focus was on the details. And details make (or break) the image.

Generative Design Variations M.6.2.2 Asynchronous loading of an XML text

This is the second and last theoretical exercise about the chapter ‘Data from the Internet.’ Here is again a brief description of the Generative Design book: ‘Loading a file, especially from the Internet usually takes much more time than the duration of a frame. However, when the XML is loaded while the program is running, program execution is interrupted constantly during that time. This is especially annoying when something is supposed to move continually on the display. In order to avoid this, a function is included in the Generative Design library with which the XML can be loaded asynchronously (ie without interrupting the program flow)’. So far the summary of this exercise. And here is the original program.

Basically, this exercise is not much different from the previous one. You download XML data from Wikipedia’s Superegg page. The XML text is displayed in the Processing Console. The difference is that during that download procedure Processing is running an animation. In addition to that, the draw block must constantly check whether the download process has already started and finished. If all the XML is downloaded, Processing displays a text in the console that the XML text is available.

If I run the program (M_6_2_02) it shows that it is not working. Nothing is displayed in the Processing console. Although the animation is running in Processing’s display window. When I’m clicking in the display window I get a comment in the console which says: “XML not loaded yet. Plus a remark [Fatal Error]: 1: 1: Premature end of file. So I suspect that this is the same mistake as in the previous program (M_6_2_01). And it is! The URL in the program begins with http://. The correct URL begins with https://. When I change that, the program works fine. First it begins with the animation. After the display window is clicked, the XML text is displayed in the console.

That‘s basically all there is to say about this exercise. I‘ve considered for a moment making a number of animations that are more complex than the one which is generated by this program. But that‘s a bit crap because the next exercise does just that. So it’s better to make variations on that exercise. I understand the principle behind this one so I leave it for what it is and I am going to concentrate on the next exercise.

Generative Design Variations M.6.2.1 The Wikipedia API

This is the first theoretical exercise of two which I need to complete to finish the last exercises of the Generative Design book. Here is a brief description of the Generative Design book: ‘In principle, all data that can be seen in a browser can also be loaded with another program and used for other purposes. If an HTML text from a web page is not required, but the data is needed in a more structured way, it is more convenient to have this information available in XML format. Luckily, this is now often the case. RSS feeds, for example, are a special XML format, and many Internet pages provide programming interfaces (so-called APIs) with which information in XML format can be specifically queried. The Wikipedia API offers a comprehensive programming interface that allows the user to both query the Wikipedia database and edit articles.’ Until here the summary of the Generative Design book. Here is the original program.

Getting data from the Internet is not new to me. I first came into contact with the Internet in 1992. I was working at Océ Research & Development. But I will not go into that any further because it is not relevant for this exercise. I usually download a text.csv or text.tsv file to use the data in data visualizations. But downloading data through an API I’ve never done before. So that is new to me.

The first thing I tried is to type in a URL from the Generative Design book Then I get a page-which is the HTML representation of the JSON format. Then I have the question: What is JSON format? JSON (JavaScript Object Notation) is a format for data exchange. JSON is based on a subset of the JavaScript programming language. JavaScript? Oh no … not again. I need XML. Anyway … JSON apparently has a lot of advantages. It is compact. People and computers can understand it easily. It maps easily to the data structures used by most programming languages (numbers, strings, booleans, nulls and arrays). Until so far, all known things. Another advantage is that almost all programming languages have features or libraries that can read or write JSON structures. That all sounds very promising. But I need XML to complete this exercise and JSON is apparently an alternative to XML. But JSON seems to be more compact and it is easier to read and write than XML. The point is now that if you’re not careful you’re ending up doing other things than completing this exercise. Then you’ll get on the path that leads to figuring out how JSON, PHP (Hypertext Pre-processor) and AJAX (Asynchronous JavaScript And XML) works. Or try Perl perhaps? So for the time being it seems not a good idea. So I stopped following that path and continued to study the This is the introduction page for the Wikipedia API.

Back to the Generative Design program M_6_2_01.pde. This program is not working. If I run it I get a [Fatal Error]: 1: 1: Premature end of file in my console. But because I have studied the Wikipedia API I suspect that this error is caused because the start of the URL in the program begins with http://. And I think the right URL should be: https://. It needs the ‘s’. https is ‘Hyper Text Transfer Protocol’ with Secure Sockets Layer (SSL). This is another protocol primarily developed with secure, safe Internet transactions in mind. So when I correct that and let the program run then I get a neat list of 27 numbered links from Wikipedia Superegg page in my Processing console. So I think this problem is now solved. I also tried other pages by requesting Superegg replacing Superegg with Generative%20Design (The% 20 comes from percent-encoding a space). I have done the same with New%20York, Steve%20Jobs, Dada and Kurt%20Schwitters. And these URL’s all work fine. All links on those pages are neatly sorted in alphabetical order and displayed in my Processing console. I think that this exercise is over. Furthermore, there are no graphics or variations of the program involved.