Rules of Summer - The App

The App version of Rules of Summer is a multimedia experiment undertaken in partnership with Melbourne's We Are Wheelbarrow, co-produced by Sophie Byrne, with whom I worked on the film The Lost Thing. So what is it exactly? Is this app any different from the book, or just the same thing on an iPad? I’m glad you asked!

There are four key elements that diffentiate the app from a simple ebook, and also from many other apps: extremely high image quality, experimental music and sound, lighting effects, and a ‘sketch mode’ showing comprehensive developmental work for the book. Below is a more detailed explanation of these unique aspects. You can also find links to the app here, and recent reviews here:

The Bizri Professional Mobile Application Reviews

School Library Journal (US)

Sydney Morning Herald (including a video preview)

In addition, the app offers many different language modes, as well as the facility for upgrades (which you’d often like to offer with a printed book!), and is easily accessible in most territories throughout the world.

Languages available so far: English, Arabic, French, German, Italian, Japanese, Korean, Portuguese, Simplified Chinese, Spanish, Traditional Chinese

Extreme Resolution

My original paintings were photographed using a special Hasselblad 200mp camera to capture very high-resolution surface detail, much higher than that visible through ordinary book reproduction (to give you and idea, images in the book are about 9,000 pixels wide, in the app they are up to 24,000). The intention was to allow the viewer to zoom in and out of the textured oil-on-canvas (originally about 80 x 70cm, so reasonably big), seeing as much as would be possible in a live gallery context – every brushstroke, scrape and drip, both deliberate and accidental! As a lover of painting myself, I always examine surfaces closely at galleries and have wanted to be able to do this with printed images too. The easy zoom of the iPad makes this possible of course. I worked closely with specialist photographer Matthew Stanton to capture the paintings using carefully controlled lighting to best represent the texture of the painted surface. Ironically, it’s the virtual medium of an iPad that best reveals a painting to be such an essentially physical object.

The original concept of Rules of Summer involved a ‘pull-out’ effect, of moving from small details to a larger context. For example, seeing a boy reach for a forbidden hors d’oeuvre at a party, and only later realizing that he is surrounded by ravenous falcons; seeing a boy step on a snail, but only later realizing there is a huge tornado bearing down behind him. These kind of amusing ‘reveals’ are not practical in printed form, but work very well on an iPad, with the added advantage of the reader being able to track and move at their own pace, and so also liberated from the controlled perspective of a film or slide-show.

Because the images are so large, a considerable amount of technical innovation was needed to compress all the data into a convenient form, and the boffins at Wheelbarrow were able to do this remarkably well (they won’t reveal their secrets and I still can’t figure out how it works!) This kind of application should be of significant interest to art students, professionals and archivists: original 2D artworks are able to be reproduced and studied in unprecedented detail, where the original is inaccessible. The luminosity of backlighting and relatively good colour calibration of an iPad also makes for highly faithful reproduction– avoiding the vagaries of physical printing. The Rules of Summer app is actually the most faithful reproduction of my work I’ve seen so far.

Music and Sound

My artwork has often been married with sound and music, from theatrical adaptations to an orchestral performance for the Red Tree composed by Michael Yezerski, who also scored my film The Lost Thing. Another composer I’ve known for some years is Sxip Shirey, an accomplished experimental musician based in New York City, particularly well known for his use of found objects to create music: everything from marbles rolling around a bowl to chicken in a deep fat fryer. I’d always wanted to work with him in some way, and Rules of Summer seemed the perfect opportunity.

Sxip’s approach was to create something that was as much a soundscape as a musical score, in part because there is no animation or ‘pace’ in my images, as I was very keen to NOT have movement within my paintings in order to preserve the timeless feeling of painted canvas, which I like so much. Yet a musical score generally moves in time, has a set duration, and can make a painting seem resistant or frozen when played against it, so creating a necessary feeling of unity between the two was no easy task.

We began the collaboration by just exchanging thoughts on the paintings, how they reminded us of certain childhood experiences as well as current influences. Sxip then experimented with a variety of forms, drafting ‘sketches’, and we exchanged many notes about what we thought worked and didn’t work, from specific sounds to problems of looping, which can feel mechanical if not handled sensitively. Some pieces were spot-on straight away, and others provoked much longer conversations and revisions. These were particularly interesting because, like any prolonged creative process, you are compelled to really think deeply about your own conceptual and emotional knowledge, to explain why one thing has more ‘truth’ than another. We often shared thoughts about growing up and interactions with family (a central theme of the story), as well as more theoretical conversations about the connection between abstract and concrete experiences of the world, from suburban Melbourne to inner-city New York.

You can go here to find out more about Sxip Shirey, and here is a link to his score for Rules of Summer as an independent release (coming soon)

Lighting Effects

Every problem is an opportunity. In spite of Sxip’s excellent aural interpretation of each scene, unfolding beautifully as the viewer navigated their way through each image, there remained a feeling of discord between kinetic sound and immobile paintings: my work felt almost too frozen by comparison. We had already decided not to include animated elements, as I personally feel that images should either be completely animated or not at all (as in the case of The Lost Thing book and film, it’s either one or the other) and I’ve never felt entirely comfortable with the partial animation I see in many apps, where only some parts of a picture are moving, I find that a confusing universe. We’d also decided not to include automated image panning or zooming, which can otherwise break that static tension, because it was important that the viewer be able to explore at their own pace. So what to do?

We came up with one possible solution: changing patterns of luminosity. This meant that there was a feeling of passing time, sympathetic to shifting sound, without involving any actual movement. At first we were a little concerned that it could end up looking like those illuminated waterfall paintings you sometimes see in Chinese restaurants, ie. still a kind of partial animation or gratuitous special effect. But we tried it out: working with the digital artists at Wheelbarrow, I broke my paintings into sections and created simple ‘light maps’, so that, for instance, areas of sky experience subtle hue shifts, metallic eyes twinkle, a giant red rabbit to pulses menacingly or a TV casts an irregular, flickering glow. The effect maintains the original surreal stillness of each image while adding a strange ambience to them, as if they are memories in the process of being retrieved. Like dreams, they are at once fixed and uncertain. The trick was keeping these graduations subtle, almost to the point of not being noticeable at first, yet visible enough to make the images ‘breathe’ and dissolve that problematic surface tension within a soundscape. I think that all worked very well in the end, and the app is able to deliver an immersive experience, somewhere between watching and reading.

Sketches

In many ways, we approached Rules of Summer as an art app more than a book app, something that would be of particular interest to people who like to draw and paint themselves. When I’ve exhibited original book illustrations, I’ve often presented my working drawings alongside them; I also compiled a number of them into my small sketchbook The Bird King because I think they are, by themselves, very interesting pieces of expression. Spontaneity, indecision and revision are integral parts of the creative process, and usually more evident in the preliminary work than the final paintings.

When users of the app have finished browsing the book in its final form, an alternative mode is unlocked whereby you can view each image as a preliminary sketch in the same high resolution, so you can closely examine all my wobbly lines and half-erased mistakes! Having viewed all of the ‘sketch mode’ a new feature unlocks, which is essentially a recreation of the kind of pin-up board I use in my studio when working on a book: a large map of different sketches and colour studies which you can freely explore by panning and zooming. The beauty of a digital folio is that there is less constraint on volume or size, so many more working images can be published than possible in a conventional book.

I’ve always been quite interested to show my working process in relation to books and films, even though I may not always like these drawings, and occasionally even find them embarrassing. The fact is that I find this part of a project the most engaging, actively brainstorming and sorting through ideas to figure out what works – the laboratory stage, which accounts for about two thirds of any project. I also enjoy looking at the sketches of other artists for the same reason, and think we can learn something from sharing the contents of our bottom-drawers and work-cupboards.

How did this project came about?

I was approached by director Tim Kently of Wheelbarrow and fellow author-illustrator Nick Bland around the time I was working on my paintings for the Rules of Summer book (then without a title): we had met previously and discussed the possibility of an ebook or app, but none of my projects seemed suitable, since they are almost always devised only with a printed book in mind. To be honest, I was also a little skeptical about ebooks and apps at the time. So much of my artistic life has been devoted to the study of static images, with a strong preference for non-digital media, and I’m especially fond of drawing and conventional literature for their silence and stillness, enlivened only by the reader’s imagination. Even after the thoroughly digital and kinetic world of making The Lost Thing as a film, I’m still very much an oil-on-canvas, ink-on-paper kind of guy.

However, Rules of Summer did seem to me to offer some opportunity for an interesting adaptation. In part, I’d always envisaged the imagery as moving from small close-up images to large landscapes, and felt like something an app could facilitate, as described above. I also saw an opportunity for introducing high-resolution reproduction, particularly in the case of this project, where my paintings were much bigger than usual and also more textural. Realistically, relatively few people will see the original artwork, even when exhibited – and exhibitions are often costly and time-consuming events, not to mention geographically limited. But a well-produced app could provide a fair substitute.

The team at Wheelbarrow agreed, and proceeded to develop the app, which involved quite a bit of research and development. Image compression was a big challenge, as the original images are huge files, yet the entire app needed to be a convenient download, about 1GB, plus sound files and other programming too! We also tried different menu formats, with an emphasis on non-linear navigation, since the book is not really a linear story; also we wanted to avoid unnecessary menus, prompts, instructions or explanations, as these can detract from an immersive, dreamlike experience.

Overall, I found the process very interesting, particularly unifying the elements of images, navigation and sound, being different to both book and film production. I see the app as quite a different object to the book, as well as a complimentary partner. Those who enjoy the book will find more to see on the app, and likely to notice a few new things. Those who enjoy the app will appreciate the experience of reading the large-format book (which never runs out of charge) as its own little universe.

Image resolution allows the viewer to see as much detail as can be seen on each original canvas. Each painting also unfolds by starting on a specific element, from which the viewer must slowly pull out to reveal the larger picture: a very interesting way of presenting a visual narrative.

Lighting transitions are created by varying the opacity of several separate layers, digitally separated out from the original paintings. In this case, adding to the impression of ominous weather.

Below: a few of the many working drawings and colour studies featured in the app.