Rules of Summer - The App

The app for iPad version of Rules of Summer is a multimedia experiment originally undertaken in partnership with Melbourne's We Are Wheelbarrow, co-produced by Sophie Byrne, with whom I worked on the film The Lost Thing. So what is it exactly? Is this app any different from the book, or just the same thing on an iPad?

There are four key elements that differentiate the app from a simple ebook, and also from many other apps: extremely high image resolution, experimental music and sound, lighting effects, and a ‘sketch mode’ showing comprehensive developmental work for the book. The app offers many different language modes: English, Arabic, French, German, Italian, Japanese, Korean, Portuguese, Simplified Chinese, Spanish, Traditional Chinese

Extreme Resolution

My original paintings were photographed using a special Hasselblad 200mp camera to capture very high-resolution surface detail, much higher than ordinary book reproduction (to give you and idea, images in the book are about 9,000 pixels wide, in the app they are up to 24,000). The intention was to allow the viewer to zoom in and out of the textured oil-on-canvas (originally about 80 x 70cm, so reasonably big), seeing as much as would be possible in a live gallery context – every brushstroke, scrape and drip, both deliberate and accidental. As a lover of painting myself, I always examine surfaces closely at galleries and have wanted to be able to do this with printed images too. The easy zoom of the iPad makes this possible of course. I worked closely with specialist photographer Matthew Stanton to capture the paintings using carefully controlled lighting to best represent the texture of the painted surface.

The original concept of Rules of Summer involved a ‘pull-out’ effect, of moving from small details to a larger context. For example, seeing a boy reach for a forbidden hors d’oeuvre at a party, and only later realizing that he is surrounded by ravenous falcons; seeing a boy step on a snail, but only later realizing there is a huge tornado bearing down behind him. These kind of amusing ‘reveals’ are not practical in printed form, but work very well on an iPad, with the added advantage of the reader being able to track and move at their own pace, and so also liberated from the controlled perspective of a film or slide-show.

Music and Sound

My artwork has often been married with sound and music, from theatrical adaptations to an orchestral performance for the Red Tree composed by Michael Yezerski, who also scored my film The Lost Thing. Another composer I’ve known for some years is Sxip Shirey, an accomplished experimental musician based in New York City, particularly well known for his use of found objects to create music: everything from marbles rolling around a bowl to chicken in a deep fat fryer. I’d always wanted to work with him in some way, and Rules of Summer seemed the perfect opportunity.

Sxip’s approach was to create something that was as much a soundscape as a musical score, in part because there is no animation or ‘pace’ in my images, as I was very keen to NOT have movement within my paintings in order to preserve the timeless feeling of painted canvas, which I like so much. Yet a musical score generally moves in time, has a set duration, and can make a painting seem resistant or frozen when played against it, so creating a necessary feeling of unity between the two was no easy task.

We began the collaboration by just exchanging thoughts on the paintings, how they reminded us of certain childhood experiences as well as current influences. Sxip then experimented with a variety of forms, drafting ‘sketches’, and we exchanged many notes about what we thought worked and didn’t work, from specific sounds to problems of looping, which can feel mechanical if not handled sensitively. Some pieces were spot-on straight away, and others provoked much longer conversations and revisions. These were particularly interesting because, like any prolonged creative process, you are compelled to really think deeply about your own conceptual and emotional knowledge, to explain why one thing has more ‘truth’ than another. We often shared thoughts about growing up and interactions with family (a central theme of the story), as well as more theoretical conversations about the connection between abstract and concrete experiences of the world, from suburban Melbourne to inner-city New York.

You can go here to find out more about Sxip Shirey, and here is a link to his score for Rules of Summer as an independent release (coming soon)

Lighting Effects

Every problem is an opportunity. In spite of Sxip’s excellent aural interpretation of each scene, unfolding beautifully as the viewer navigated their way through each image, there remained a feeling of discord between kinetic sound and immobile paintings: my work felt almost too frozen by comparison. We had already decided not to include animated elements, as I personally feel that images should either be completely animated or not at all (as in the case of The Lost Thing book and film, it’s either one or the other) and I’ve never felt entirely comfortable with the partial animation I see in many apps, where only some parts of a picture are moving, I find that a confusing universe. We’d also decided not to include automated image panning or zooming, which can otherwise break that static tension, because it was important that the viewer be able to explore at their own pace. So what to do?

We came up with one possible solution: changing patterns of luminosity. This meant that there was a feeling of passing time, sympathetic to shifting sound, without involving any actual movement. At first we were a little concerned that it could end up looking like those illuminated waterfall paintings you sometimes see in Chinese restaurants, ie. still a kind of partial animation or gratuitous special effect. But we tried it out: working with the digital artists at Wheelbarrow, I broke my paintings into sections and created simple ‘light maps’, so that, for instance, areas of sky experience subtle hue shifts, metallic eyes twinkle, a giant red rabbit to pulses menacingly or a TV casts an irregular, flickering glow. The effect maintains the original surreal stillness of each image while adding a strange ambience to them, as if they are memories in the process of being retrieved. Like dreams, they are at once fixed and uncertain. The trick was keeping these graduations subtle, almost to the point of not being noticeable at first, yet visible enough to make the images ‘breathe’ and dissolve that problematic surface tension within a soundscape. I think that all worked very well in the end, and the app is able to deliver an immersive experience, somewhere between watching and reading.

Sketches

In many ways, we approached Rules of Summer as an art app more than a book app, something that would be of particular interest to people who like to draw and paint themselves. When I’ve exhibited original book illustrations, I’ve often presented my working drawings alongside them; I also compiled a number of them into my small sketchbook The Bird King because I think they are, by themselves, very interesting pieces of expression. Spontaneity, indecision and revision are integral parts of the creative process, and usually more evident in the preliminary work than the final paintings.

When users of the app have finished browsing the book in its final form, an alternative mode is unlocked whereby you can view each image as a preliminary sketch in the same high resolution, so you can closely examine all my wobbly lines and half-erased mistakes! Having viewed all of the ‘sketch mode’ a new feature unlocks, which is essentially a recreation of the kind of pin-up board I use in my studio when working on a book: a large map of different sketches and colour studies which you can freely explore by panning and zooming. The beauty of a digital folio is that there is less constraint on volume or size, so many more working images can be published than possible in a conventional book.

I’ve always been quite interested to show my working process in relation to books and films, even though I may not always like these drawings, and occasionally even find them embarrassing. The fact is that I find this part of a project the most engaging, actively brainstorming and sorting through ideas to figure out what works – the laboratory stage, which accounts for about two thirds of any project. I also enjoy looking at the sketches of other artists for the same reason, and think we can learn something from sharing the contents of our bottom-drawers and work-cupboards.

Overall, I found the process very interesting, particularly unifying the elements of images, navigation and sound, being different to both book and film production. I see the app as quite a different object to the book, as well as a complimentary partner. Those who enjoy the book will find more to see on the app, and likely to notice a few new things. Those who enjoy the app will appreciate the experience of reading the large-format book (which never runs out of charge) as its own little universe.