The impact of digital technologies on the production and consumption of moving images 2007-2016
3. High resolution research 2007 - 2010
When applying for an AHRC Creative Research Fellowship in 2006, I argued that there were no practitioner-led research projects into the effects of high definition video underway in UK HEIs:
‘The aim of this project is to investigate in practice and in theory what is happening to the audience gaze as it shifts from the analogue, to the digital, to the higher resolution. This impending change has focused my artistic and technical concerns into the following question:
My application entitled ‘High Definition Installations and Single Screen Pieces: An Investigation into the Actual, the Virtual and the Hyper Real’ outlined a 36 month practice as research process and methodology to create works in HD and examine audience response. On being awarded the AHRC fellowship I then fulfilled my proposition and made a series of practice as research outputs, created evaluation procedures and published peer-reviewed articles and brought results to conference. Effectively I identified 4 strategies for revealing the effects of resolution:
In all of the initial research my motive was to reinvent display practices within the medium because cinema had set out it’s stall in a particular way, which flowed on from the theatre by having a ‘proscenium arch’ – that is we the audience sat and looked in one direction - and very early on I realised I had to ‘take the screen off the wall’. This was due to it’s location then as now as a cinema or television screen, which itself carried with it the associations that rendered it familiar. I realised that to re-present the familiar I had to provoke a sense of the unfamiliar in the belief that the potential aspect of ‘unheimlich’ or ‘the uncanny’ might reveal something of the changes between standard definition (which we had accommodated in our familiar acceptance of moving image) and high definition, which itself was defined in relation to the standard delivered by the limitations of early twentieth century manufacturing competencies. In fact after this period of experimentation with artefacts, installation art became dependent on what I had chosen as a standard for experimentation with research artefacts – projecting images onto all surfaces rather than vertically situated surfaces. My screens would therefore be the very objects I would ask people to look at to see what kind of engagement I could obtain if I increased the levels of resolution and therefore the verisimilitude of its representation as if real. This would necessarily deal with the hyper-real which could exist within the liminal boundary where suspension of disbelief might become activated. This first AHRC fellowship therefore included the reference to ‘the Actual, the Virtual and the Hyper Real’.
For example if I looked at a dinner scene projected onto a table, what level of resolution would be convincing to an audience beginning with European standard definition as my measure (720 x 576 pixels)? At that level, photographed from overhead a fork then projected at life size only revealed its prongs as a grey blur. In HD at 1920 x 1080 (or 1080p) – or roughly four times the resolution the individual prongs were revealed. What might this mean?
In late 2006 I proposed the following:
“I have devised a series of works, to be made with High Definition equipment for public exhibition. The series comprises both installations and single screen works, representing two sides of the increasingly blurred line between visual art and cinema. They are linked by a concern with place and the reality (or virtuality) of place. My intent is to bring real places (landscapes and cities) into an internal space on a single screen; and in contrast, to project virtual objects in an installation setting back on to a real object. Thus through the series I aim to draw out a critical reflection on the comparative nature of the two modes of configuring “reality”. The installations will be characterised by a blurring of the lines between actual and virtual. I am keen to explore the unveiling of the photographic moment: that is, the redefinition of the ordinary into the extraordinary by the act of segregating that moment and showing it using the tactic of extended duration”.
This last point become increasingly important as I further investigated the relationship between duration and resolution. Conrad Hall the great Cinematographer (American Beauty and The Road to Perdition amongst many other films) had identified the photographic moment as that which exemplified the most pertinent gesture of the photograph by capturing the essential aesthetic element of the subject (this as opposed to Cartier Bresson’s journalist definition which revolved around what he called ‘the definitive moment’ or it’s most compelling story). Hall argued that in every frame of a movie the cinematographer’s task is to identify that essential element even in ‘dead space’ (often in a pan between two faces the camera might travel across a not soecific area of the image that technicians call ‘dead space’. Hall argues that the cinematographer’s job was to bring out the highest aesthetic elements in that journey – hence he named it the photographic moment because if examined as a photograph, the highest aesthetic would be revealed in every single frame). Also Bill Viola had rightfully proposed a tactic buried in his now famous aphorism which was also familiar to the Director Andrei Tarkovsky before him:
‘Duration is to Consciousness as Light is to the eye”.
What is being described here is as the eye is bathed by light and so becomes activated, then the mind can also be bathed with the quality of ‘duration’. Simply put, the act of agreeing to pay more attention to an image than the audience was used to in either cinema or television would deliver additional benefits. I started to think through some combinations of ideas:
All of the above of course has its equivalence 10 years later in the use of a phone or computer where the brain is said to achieve a ‘dopamine hit’ every time the attention is pushed to demand another ‘high’. Except of course, the above argues that the pleasure is different and not reliant on the rise and fall of a desire for another high.
To return to Viola: 'Duration is to consciousness as light is to the eye.' But High Definition can deliver not just duration but articulation. So it struck me that we might now remember how increased resolutions could affect how and what we see and therefore re-formulate his observation like this:
'Definition is to consciousness – as luminosity is to the eye.'
I then began to realize that because Resolution is to consciousness as Luminance is to the eye, consciousness is activated in a qualitative way and it’s as if these two twin facets of the same thing are equivalent. Therefore one could say:
Time = Resolution.
Even now mainstream film and TV has the four second rule, that proposes that no image should remain on screen for longer than 4 seconds lest the audience lose interest. This is of course is about film grammar rather than a constituent quality of the image itself. It's about whisking away every image to create a sent of compulsion of narrative.
Much later in my research I used the tactic of engaging participants in knowledge exchange via durational experiences. Immersive Learning Environments were defined by single discipline focused events that asked all levels of one specific discipline (from student to professor and professional) join together in research.
Later I was to develop this further into an exercise which assembled people from different disciplines in Advanced Innovation Laboratories – and this was to include not only the specialists but the generalist in the form of the audience itself, (when being taught to see depth in an HDR lab at the Arnolfini in 2014 which spoke to the specialists, that the research was effectively, successful). Also, even towards the end of the research period when speaking with the highest level professionals of IMAGO, the society which I was inducted onto the education panel of (being the organisation that represented all of the worlds Cinematographic Societies) and at the subject areas’ nexus, the Science and Technology Committee of the Academy of Motion Pictures in Los Angeles (where I was asked to participate primarily in the development of Higher Dynamic Range studies). At this last stage during my presentation I asked the committee to suspend its boundary of disbelief and witness spatial qualities within a medium which had not been designed to encode the extra dimension of depth (as had the function of binocular stereopsis within 3D cinema).
Often one of the conditions of the presentation of the medium (i.e. television or Cinema - the seductive ‘Silver Screen’) is of having the intent to create a consuming spectacle - so paying attention to a particular constituent element of the medium - to duration itself – might then reveal the essential and constituent parts that came together in the understanding of Digital Cinematography as a primary expository technology of new digital media.
At the beginning of the research therefore, I worked through the effects of increasing resolution such that by the end of that first fellowship I could then formally recommend in peer-reviewed articles two main propositions. The first with regard the relationship between resolution and durational attention:
The second with regard how not only professionals but academics and students of the discipline could wield the new technology effectively and appropriately:
At the end of this first fellowship (2010) I organised symposia7, an exhibition and a conference that paired professionals and academics to create a language sympathetic to both. I documented these and these then augmented the fourth PARP to rebalance its focus from online resource to account of impact of research. The next initiative in research would continue to rebalance this PARP with the addition of a further element within the portfolio with Higher Dynamic Range Research such that exposition and exchange would be part of the generation of new knowledge.
Knowledge Exchange and experiments in Higher Dynamic Range 2010 - 2012
Between 2010 and 2012 I was awarded a second AHRC Fellowship, (again with UoB). This was originally called a Knowledge Transfer Fellowship and was situated in the same department and was to focus on transferring the results of the first fellowship with industry and academia in traditional ways with the addition of various public exhibitions and presentations.
But this then changed when academia began to rename its behaviour and my fellowship became a Knowledge Exchange Fellowship. Academia had begun to realise that transfer is assertive whilst exchange is reflexive and I went through that conceptual shift at the same time. Within this fellowship I innovated strategies to clarify what knowledge could be gained from the original research that involved the creation of 20 workshops with the following aims:
These workshops engaged professionals, undergraduates, graduate students and academics and involved innovation of methods of fast dissemination of complex new knowledge. I maintained a set of before and after responses to online questionnaires to establish the efficacy of the methods used, and then published The Practice of Knowledge Exchange7 an article that debated the nature of research, how not only knowledge is exchanged but also how new methods and new knowledge may be innovated (which summates the information gathered from the 20 workshops undertaken). This laid the foundations for the ideas mentioned earlier: Immersive Learning Environments (and later its developed state as an Advanced Innovation Laboratory).
I then moved to Department of Engineering at Bristol as a Senior Research Fellow to explore deeper aspects of the construction of the image from an entirely different perspective. An additional question arose for me:
Suddenly I was my own test case for the question above – not in terms of ignorance of the subject area as I had engineering knowledge, but in fact, in terms of the potential differing forms of expression of what I actually knew. Might my own cultural differences with my engineering colleagues, coming from an arts and humanities background, open new areas of research? (This question was to grow as time went on and eventually manifested in an Advanced Innovation Laboratory in March 2014).
During this period Professor David Bull of Department of Engineering at Bristol University had approached me to lead Higher Dynamic Range research (HDR) that was ongoing in Faculty of Engineering since 2007. Professor Bull led the Bristol Vision Institute (BVI)8, mentioned earlier, which welcomed different scientific practices to contribute to an understanding of the mechanics, the physiology, and the psychological effects of the construction of an improved moving image ‘container.’
Aim of Research
When applying for an AHRC Creative Research Fellowship in 2006, I argued that there were no practitioner-led research projects into the effects of high definition video underway in UK HEIs:
‘The aim of this project is to investigate in practice and in theory what is happening to the audience gaze as it shifts from the analogue, to the digital, to the higher resolution. This impending change has focused my artistic and technical concerns into the following question:
- How will High Definition Imaging affect the nature of art and entertainment from the point of view of both practitioners and audiences?’
My application entitled ‘High Definition Installations and Single Screen Pieces: An Investigation into the Actual, the Virtual and the Hyper Real’ outlined a 36 month practice as research process and methodology to create works in HD and examine audience response. On being awarded the AHRC fellowship I then fulfilled my proposition and made a series of practice as research outputs, created evaluation procedures and published peer-reviewed articles and brought results to conference. Effectively I identified 4 strategies for revealing the effects of resolution:
- By concentrating on the representation of the domestic objects around us
- By looking at objects that are at a distance from us
- By looking at ourselves within that broader landscape
- By recording the views of those leading in the design and use of this emerging technology
In all of the initial research my motive was to reinvent display practices within the medium because cinema had set out it’s stall in a particular way, which flowed on from the theatre by having a ‘proscenium arch’ – that is we the audience sat and looked in one direction - and very early on I realised I had to ‘take the screen off the wall’. This was due to it’s location then as now as a cinema or television screen, which itself carried with it the associations that rendered it familiar. I realised that to re-present the familiar I had to provoke a sense of the unfamiliar in the belief that the potential aspect of ‘unheimlich’ or ‘the uncanny’ might reveal something of the changes between standard definition (which we had accommodated in our familiar acceptance of moving image) and high definition, which itself was defined in relation to the standard delivered by the limitations of early twentieth century manufacturing competencies. In fact after this period of experimentation with artefacts, installation art became dependent on what I had chosen as a standard for experimentation with research artefacts – projecting images onto all surfaces rather than vertically situated surfaces. My screens would therefore be the very objects I would ask people to look at to see what kind of engagement I could obtain if I increased the levels of resolution and therefore the verisimilitude of its representation as if real. This would necessarily deal with the hyper-real which could exist within the liminal boundary where suspension of disbelief might become activated. This first AHRC fellowship therefore included the reference to ‘the Actual, the Virtual and the Hyper Real’.
For example if I looked at a dinner scene projected onto a table, what level of resolution would be convincing to an audience beginning with European standard definition as my measure (720 x 576 pixels)? At that level, photographed from overhead a fork then projected at life size only revealed its prongs as a grey blur. In HD at 1920 x 1080 (or 1080p) – or roughly four times the resolution the individual prongs were revealed. What might this mean?
In late 2006 I proposed the following:
“I have devised a series of works, to be made with High Definition equipment for public exhibition. The series comprises both installations and single screen works, representing two sides of the increasingly blurred line between visual art and cinema. They are linked by a concern with place and the reality (or virtuality) of place. My intent is to bring real places (landscapes and cities) into an internal space on a single screen; and in contrast, to project virtual objects in an installation setting back on to a real object. Thus through the series I aim to draw out a critical reflection on the comparative nature of the two modes of configuring “reality”. The installations will be characterised by a blurring of the lines between actual and virtual. I am keen to explore the unveiling of the photographic moment: that is, the redefinition of the ordinary into the extraordinary by the act of segregating that moment and showing it using the tactic of extended duration”.
This last point become increasingly important as I further investigated the relationship between duration and resolution. Conrad Hall the great Cinematographer (American Beauty and The Road to Perdition amongst many other films) had identified the photographic moment as that which exemplified the most pertinent gesture of the photograph by capturing the essential aesthetic element of the subject (this as opposed to Cartier Bresson’s journalist definition which revolved around what he called ‘the definitive moment’ or it’s most compelling story). Hall argued that in every frame of a movie the cinematographer’s task is to identify that essential element even in ‘dead space’ (often in a pan between two faces the camera might travel across a not soecific area of the image that technicians call ‘dead space’. Hall argues that the cinematographer’s job was to bring out the highest aesthetic elements in that journey – hence he named it the photographic moment because if examined as a photograph, the highest aesthetic would be revealed in every single frame). Also Bill Viola had rightfully proposed a tactic buried in his now famous aphorism which was also familiar to the Director Andrei Tarkovsky before him:
‘Duration is to Consciousness as Light is to the eye”.
What is being described here is as the eye is bathed by light and so becomes activated, then the mind can also be bathed with the quality of ‘duration’. Simply put, the act of agreeing to pay more attention to an image than the audience was used to in either cinema or television would deliver additional benefits. I started to think through some combinations of ideas:
- Engagement is an active property within the act of the gaze
- Attention is therefore a kind of energy (or an activator of such)
- Within engagement, sometimes time appears to cease – or in fact the nature of surging forward with interpretation (a more frontal lobe and a narrative based compulsion) might be forestalled, such that the mind comes to rest and finds a different pleasure.
- The resting gaze could be said to be equivalent to how much resolution an image had - if so it may also be equivalent (if thought about mathematically) as time engaged, and so time could be said to equal resolution.
All of the above of course has its equivalence 10 years later in the use of a phone or computer where the brain is said to achieve a ‘dopamine hit’ every time the attention is pushed to demand another ‘high’. Except of course, the above argues that the pleasure is different and not reliant on the rise and fall of a desire for another high.
To return to Viola: 'Duration is to consciousness as light is to the eye.' But High Definition can deliver not just duration but articulation. So it struck me that we might now remember how increased resolutions could affect how and what we see and therefore re-formulate his observation like this:
'Definition is to consciousness – as luminosity is to the eye.'
I then began to realize that because Resolution is to consciousness as Luminance is to the eye, consciousness is activated in a qualitative way and it’s as if these two twin facets of the same thing are equivalent. Therefore one could say:
Time = Resolution.
Even now mainstream film and TV has the four second rule, that proposes that no image should remain on screen for longer than 4 seconds lest the audience lose interest. This is of course is about film grammar rather than a constituent quality of the image itself. It's about whisking away every image to create a sent of compulsion of narrative.
Much later in my research I used the tactic of engaging participants in knowledge exchange via durational experiences. Immersive Learning Environments were defined by single discipline focused events that asked all levels of one specific discipline (from student to professor and professional) join together in research.
Later I was to develop this further into an exercise which assembled people from different disciplines in Advanced Innovation Laboratories – and this was to include not only the specialists but the generalist in the form of the audience itself, (when being taught to see depth in an HDR lab at the Arnolfini in 2014 which spoke to the specialists, that the research was effectively, successful). Also, even towards the end of the research period when speaking with the highest level professionals of IMAGO, the society which I was inducted onto the education panel of (being the organisation that represented all of the worlds Cinematographic Societies) and at the subject areas’ nexus, the Science and Technology Committee of the Academy of Motion Pictures in Los Angeles (where I was asked to participate primarily in the development of Higher Dynamic Range studies). At this last stage during my presentation I asked the committee to suspend its boundary of disbelief and witness spatial qualities within a medium which had not been designed to encode the extra dimension of depth (as had the function of binocular stereopsis within 3D cinema).
Often one of the conditions of the presentation of the medium (i.e. television or Cinema - the seductive ‘Silver Screen’) is of having the intent to create a consuming spectacle - so paying attention to a particular constituent element of the medium - to duration itself – might then reveal the essential and constituent parts that came together in the understanding of Digital Cinematography as a primary expository technology of new digital media.
At the beginning of the research therefore, I worked through the effects of increasing resolution such that by the end of that first fellowship I could then formally recommend in peer-reviewed articles two main propositions. The first with regard the relationship between resolution and durational attention:
- that a four times increase in resolution produces twice the length of audience engagement5
The second with regard how not only professionals but academics and students of the discipline could wield the new technology effectively and appropriately:
- by the creation of a set of rules for the definition and practice of Digital Cinematography6.
At the end of this first fellowship (2010) I organised symposia7, an exhibition and a conference that paired professionals and academics to create a language sympathetic to both. I documented these and these then augmented the fourth PARP to rebalance its focus from online resource to account of impact of research. The next initiative in research would continue to rebalance this PARP with the addition of a further element within the portfolio with Higher Dynamic Range Research such that exposition and exchange would be part of the generation of new knowledge.
Knowledge Exchange and experiments in Higher Dynamic Range 2010 - 2012
Between 2010 and 2012 I was awarded a second AHRC Fellowship, (again with UoB). This was originally called a Knowledge Transfer Fellowship and was situated in the same department and was to focus on transferring the results of the first fellowship with industry and academia in traditional ways with the addition of various public exhibitions and presentations.
But this then changed when academia began to rename its behaviour and my fellowship became a Knowledge Exchange Fellowship. Academia had begun to realise that transfer is assertive whilst exchange is reflexive and I went through that conceptual shift at the same time. Within this fellowship I innovated strategies to clarify what knowledge could be gained from the original research that involved the creation of 20 workshops with the following aims:
- Create points of knowledge exchange that would produce debate on the knowledge revealed by my AHRC Fellowship
- Formulate a means of evaluating who would attend and what they knew of the field before coming
- Formulate a means of evaluating what they learned during the exchange and a reliable measurement of that increase in knowledge
These workshops engaged professionals, undergraduates, graduate students and academics and involved innovation of methods of fast dissemination of complex new knowledge. I maintained a set of before and after responses to online questionnaires to establish the efficacy of the methods used, and then published The Practice of Knowledge Exchange7 an article that debated the nature of research, how not only knowledge is exchanged but also how new methods and new knowledge may be innovated (which summates the information gathered from the 20 workshops undertaken). This laid the foundations for the ideas mentioned earlier: Immersive Learning Environments (and later its developed state as an Advanced Innovation Laboratory).
I then moved to Department of Engineering at Bristol as a Senior Research Fellow to explore deeper aspects of the construction of the image from an entirely different perspective. An additional question arose for me:
- Can a person who is apparently ignorant of the deeper levels of information and wisdom currently held of a subject area creatively contribute to the expansion of the very same knowledge base?
Suddenly I was my own test case for the question above – not in terms of ignorance of the subject area as I had engineering knowledge, but in fact, in terms of the potential differing forms of expression of what I actually knew. Might my own cultural differences with my engineering colleagues, coming from an arts and humanities background, open new areas of research? (This question was to grow as time went on and eventually manifested in an Advanced Innovation Laboratory in March 2014).
During this period Professor David Bull of Department of Engineering at Bristol University had approached me to lead Higher Dynamic Range research (HDR) that was ongoing in Faculty of Engineering since 2007. Professor Bull led the Bristol Vision Institute (BVI)8, mentioned earlier, which welcomed different scientific practices to contribute to an understanding of the mechanics, the physiology, and the psychological effects of the construction of an improved moving image ‘container.’
Aim of Research
- Create a viable method of both capturing and displaying higher dynamic range images
Moving Image Arts Research is concerned with exploring the histories, theories, technologies, cultures and politics of moving image art production, interaction and reception.
|
|
Interactive Region Hot Spots
small experiments in connectivity and how drama or documentary might be enabled |