New visualisation of music

In recent months I have worked on a new approach for visualising music. Already since a long time I had been dissatisfied with the way that media players show graphics to music: it is often just based on a raw analysis of the audio, showing a spectrogram, and then some random spatial animation. In most cases there is hardly a connection between what is seen on the screen to what the music is actually playing, except that the rhythm of the music is reflected in some of the motion.

But I wanted a visualisation which is true to the music and contains all music elements in it. And I found one: a transformation which is based on a display of the rhythmic (repetitive) properties of the music. My paper about this will be presented in march 2012 at the workshop "Music, Mind, Invention" in Ewing, New Jersey.

Here just a brief summary - details will be seen in the paper: the 2D piano roll plot which is used in music sequencers for showing / editing music notes over a timeline, contains all relevant information about the music. Treating this piano roll plot as an image can then lead to a visualisation which contains all the musical elements - and this transformation is even reversible, so that the visuals can be re-translated into music.

FP7 Call 8 Info Day in Luxembourg

In July, the new Call 8 of the European Framework Programme 7 (FP-7) has been launched. On Wednesday, 5.October 2011, there will be an information day held in Luxembourg.

The topics to be dealt at this meeting are:

• Adaptive learning systems - Objective 8.1.a
• Educational technologies for science, technology and maths - Objective 8.1.b
• Advanced solutions for fast and flexible learning at the workplace (targeting, in particular, SMEs) - Objective 8.1.c
• Creativity in learning processes - Objective 8.1.d
• Exploratory activities and networks- Objective 8.1.e

The deadline for proposal submissions to this call is 17.January 2012.

Visitor from Austria: Gerhard Schwed

Today, Gerhard Schwed from the Danube University Krems (Austria), Head of e-Learning, arrived at LeedsMet. He will stay here for the next two weeks in an ERASMUS-funded exchange visit, for sharing eLearning expertise with our teams at LeedsMet.

ISMAR 2011 in Basel: 26.-29.October

This year, the annual ISMAR Conference (International Symposium on Mixed and Augmented Reality) will take place in Switzerland: in Basel, from 26.-29.October 2011. Since the time when this conference was founded 13 years ago, in 1998, as the "International Workshop on Augmented Reality (IWAR)", it has established itself as the world's leading conference on Mixed and Augmented Reality.

I was fortunate enough to have been involved for many years in this very successful conference, and I am also glad that this year I could help in selecting the papers to be published, as one of the 12 area chairs. Next Monday is the deadline for submission of the final camera-ready papers - there were many excellent submissions, but the review criteria were also very strict.

Unfortunately I will not be able to participate this year at ISMAR 2011, but I wish the organizers the very best! This shall be an excellent conference, and the participants will surely enjoy their attendance.

The EuroPLOT project

My colleague at LeedsMet, Prof Janet Finlay, has started her sabbatical year this September, and after that she will leave LeedsMet to pursue other things in life. I wish her the very best for this move!

As a result, I have now "inherited" leading the project "EuroPLOT", which she has initiated and led since last year. This project is devoted to the development of "persuasive learning objects and technologies" (PLOTs). LeedsMet's role is here to lead the project, to contribute to all the work packages, and to lead the dissemination activities. For this project, I began to expand some of the online activities: we set up a blogger page and a page on Facebook, and opened a Twitter account.

Since August 2011 we had a few internal meetings at LeedsMet, with the next teleconference meeting coming up next week. Every 6 months there is also a face-to-face meeting; the next one in November will take place in Hradec Králové, Czech Republic.

The project is moving towards the end of its first year, with two more years after that. It is funded by the European Commission Lifelong Learning Programme (Project ref: 511633-LLP-1-2010-1-UK-KA3-KA3MP).

Another academic year begins

It has been exactly to the date one year, when I posted my last entry into this blog here. For some reason, I did neglect writing here. Not because of a lack of things to report, but because I was busy with actually doing work, rather than writing about it. But I want to restart posting here, to give an account and an insight into my work at LeedsMet. As one year ago, also this year around this time the new semester is starting. Things are still quiet, next week the International students will come, a week after this all other students will have their induction.

Back for the New Semester

On 1.September I returned from my holiday into my office. The email inbox had closed due to overflow, so the first item to do was to save the emails with the largest attachments in a local folder.

Overall the first days after the summer break were quiet: no students are here yet, the campus appears empty. However, there is a lot of work going on in the offices: staff prepare their lectures and assignments for the coming semester, the time table is being setup, and the curriculum for the semester is finalised.

In order to cope with the variety of projects in which I am involved, I have devised a new method handling the often changing tasks, requirements, and ideas for those projects: somewhat based on "agile software development". It involves a patch of paper and handwritten notes. I hope to be able to create a software for this approach eventually.

Upcoming Presentations at Leeds Met Research Day

On 1. July, the Research and Enterprise conference for LeedsMet members of staff will take place at the Headingley Campus. I will give two workshops: in the morning I will talk, together with Prof Rod King, Prof Tony Parson, and Andrew Marran about the "Proof of Concept" fund, which has supported several projects at Leeds Met with the prospect of commercialisation. In the afternoon I will hold a workshop about the European FP-7 program and will describe what opportunities there are for research and development funding.

Preparing Major Bid

In recent weeks I have gathered momentum for preparing a major bid for funding, together with several partner institutions across Europe. This is quite an effort: the partners must be approached with an idea that is already more or less feasible, as a starting point for further discussions. Now the concrete milestones, work plan, and deliverables must be worked out in collaboration with these partners, and a consistent project schedule and a reasonable resource allocation must be developed before the deadline. A lot of work is still ahead in this...

Crossover Business Lab

I was fortunate enough to be able to participate in a great event which took place in Leeds from 25.-26.February 2010: the Crossover Business Lab. Frank Boyd had organised this event, with speakers who presented case studies and held workshops. One could learn a lot about business development and venture investing. This knowledge is crucial when setting up businesses and trying to commercialise some developments from within the university. The speakers were enthusiastic and conveyed very well the issues that businesses face when then move from one stage to the next.

"Virtual Runner" Demonstration

Since summer 2007, we have been working in a collaboration between the Leeds Met Carnegie Faculty and the Faculty Innovation North in developing a learning game, which allows students to intuitively learn about physiological processes in a playful way. The second prototype of this simulation "Virtual Runner" has now publicly been demonstrated for the first time at the Venturefest 2010 in York on Wednesday, 10.February. At this occasion we received a lot of positive feedback.

We also had the opportunity to present this demonstration to the Leeds Met Governors during their meeting last Friday (12.February 2010), and they were very positive about this project.

During the first development phase, the project had been developed through funding from the HEFCE Teaching Quality Enhancement Fund (TQEF). The first prototype had been developed by Kooji Creative who are a student-owned (former graduates of Leeds Met Innovation North) organisation. The 2nd prototype which was the subject of the latest demonstrations has been developed with help from the Yorkshire Proof of Commercial Concept Fund, matched by funding from the Higher Education Innovation Fund (HEIF). Innovation North graduate Abraham Smith was the key developer of the software of this prototype.

We intend to offer more information at the Virtual Runner website.

Visit of Peter Baumgartner

Prof Peter Baumgartner from the Danube University Krems (DUK) visited our faculty Innovation North during the past two days. Since 2005, several students from DUK had registered at Leeds Met for a PhD degree, and the first students are approaching the completion of their degree. In our meetings with Peter we discussed the terms for future collaboration in teaching and also in research activities.

Peter was very impressed by the interdisciplinary activities which we demonstrated within our faculty and in collaborative work with other faculties, and by the 24/7 access to our library.

Unfortunately the weather here in Leeds was not very hospitable: rain on Tuesday, snow sleet on Wednesday, and today thick fog which led to the cancellation of his flight. He will have a very long journey back to Krems tonight.

Interactive Arts Display

In the past few weeks I have worked on writing software for an interactive arts display. The main concept is that a video camera monitors the visitors of the arts display, which is a projection of a video film showing the artist Lee Gascoyne creating an artwork (see his blog at http://artists-water.blogspot.com/. The replay of this video is influenced by the motion of the visitors, which is captured by the camera: only of the audience does not move, the video recording will be played properly.

The technical principle of this system is as follows: The video capture software captures a reference image and computes the "difference" of each subsequent video image, pixel by pixel, in RGB. Based on this difference the video replay is being disturbed, and the video jumps ahead or backward. The implementation of this interaction has been done controlling the video player Winamp: this player can be controlled from another software by sending Windows messages. In order to be able to place the video replay on a different PC than the video capture, a socket connection between these two programs has been implemented: the camera capture software runs a server, which is sending out the pixel change value to each client after connection.

In our tests today at the Barnsley University Campus of Huddersfield University (where Lee is a BA Hons student), Senior Computing Officer Alistair Reid-Pearson (who is a Leeds Met Alumni from Innovation North) set up a PC for running all the software components. There were some issues regarding downloading the .NET framework 3.5, but the system ran flawlessly on another PC which had a different image of the OS installation.

The art installation will be shown at the exhibition of "The Dearne Project" which will be opened on 5.February by Councillor Newman at the Emergency Pod 1, 14-16 Mayday Green, Barnsley, at 6:30pm. This exhibition is a part of The Rivers Movement.

European FP-7 - 6th Call

In November the European Commission has published their 6th call in the 7th Framework Programme (FP-7). This call has a variety of topics, ranging from robotics to cultural heritage digitisation: see the CORDIS web site with the details of the call.

I have posted three profiles on the CORDIS partner search site (related to Heritage Digitisation, Artificial Creativity, and general expertise) and I encourage anyone interested in joining a consortium to have a look there. For searching, no registration is required. For posting a profile and for replying to existing profiles you need to register on that site.

Interview by Times Educational Supplement

Today around noon I received a phone call from Dave Matthews from the Times Educational Supplement (TES). At first I thought I would be asked about Leeds Met and the changes it had gone through the past year, and I was not really prepared to talk about that. But fortunately, the topic of conversation was "Augmented Reality (AR)". The interviewer wants to write a column in the TES in the next days or so. I was asked about the history of AR, and I mentioned the AR work of Boeing in the early 1990s. Furthermore, I could recall that the first prototypes of mobile AR applications were presented at ISMAR sometime in 2002 or 2003; I remember seeing one of those handheld Compaq iPAQs with a camera and a graphical overlay of kitchen furniture. I was not sure when the first AR app on the iPhone came out... but AR has only made it into the mainstream awareness during the past year.

I will keep my eyes open for the article in the TES.

Talk at St. Aidan's Church of England High School, Harrogate

On 3.December I was invited by Marian Farrar, Business Liaison Officer of the St. Aidan's C of E High School in Harrogate, to give a talk about a Science and Technology topic suitable for the 6th form. I chose the topic "Towards Artificial Intelligence – How Computers 'See' the World". First I introduced the general concept of imaging, illustrating how artists during the Renaissance period discovered the perspective mapping and used it in their paintings. Computer vision basically needs to invert the mapping equation, to recover the 3D space around the camera from 2D image series. This is done by feature detection and tracking, employs image processing methods on the digitised image sequences. As a result, computer vision is able to extract enough information from the visual sensors for performing specific tasks. To illustrate this, I showed videos of cars performing automatic driving, and I talked about my contribution to visual road recognition and the US DARPA Grand Challenge. The students were very attentive, and I hope that I was able to stimulate an interest in this very important technology of Computer Vision which will have a significant impact in the next few decades.

Evaluator for the European Commission

Since the year 2000 I have been doing this: working as an evaluator / reviewer / auditor for the European Commission. The European Union awards a large amount of research funding to European consortia, and they need reviewers (experts) to judge the quality. There are usually many more proposals than there is funding available, and so these experts have the task to put the incoming proposals in a ranking - the top ranked proposals will then receive the funding.

This evaluation activity is done several times a year, and each time a new call is launched for proposals, the European Commission organises these reviews and invites experts all across Europe to participate in those reviews/evaluations. For the evaluators, this is a great honour to be invited, and it is also beneficial: they get to see the very latest state of research interests, and they can expand their network with other evaluators. Naturally all the proceedings are treated confidentially, so no evaluator can use the information obtained from reading those proposals.

Information about the current EU workprogrammes (Framework Programme 7 = FP7) is on the CORDIS website. This site has also a section where everyone with any considerable professional or academic expertise can register as expert. It is recommended to frequently update the profile there, because that is the expert repository from where the European Commission selects the evaluators so that their expertise matches the requirements for evaluating projects and proposals.

When the Commission invites the experts, they first send out an email for probing the availability at a given time. This is usually 4-6 months before the event and gives them an overview on how many experts would be available. About 3 months before the evaluation event an official letter will be sent with an invitation to participate in the evaluation. The expert then has to sign a contract, declaring any conflict of interest and complying with the confidentiality requirements. This then constitutes a binding agreement, and the expert can then book the travel - this will be reimbursed by the Commission later.

About 2 weeks before the actual meeting (which usually takes place in Brussels), the experts are given access to the material that they are to evaluate. Either the proposals are being sent by postal mail, or they are made available online. It is then the task of the evaluator to do a thorough review of the material and give a score, based on set criteria. These criteria may differ for each call, as the procedures slightly change over the years and are also different for each discipline. Th scores are then submitted back to the Commission before the meeting.

The meeting where all the evaluators come together can last a few days or a whole week. The purpose there is to come to a consensus: for each proposal, a number of reviewers (typically 3) provides a score, and these score may differ. In scheduled consensus meetings, these evaluators now discuss why they believe that their score is appropriate. They need to be prepared to change their score, because for the final ranking, the score is not simply the average of all evaluator scores, but is the score to which each evaluator agrees. There are mechanisms in place in case the evaluators have very stubborn opinions - another evaluator would join, and then they would try to reach consensus. They all sign the score sheet in the end, indicating their agreement of the score. One of them is assigned to be a rapporteur, writing the notes and comments into the final form that will be forwarded later to the applicants for the funding.

After all the individual project consensus meetings have been held, a final meeting with all evaluators takes place. The goal here is to achieve a ranking of all the proposals. Since not every evaluator will have seen every proposal, there is still a chance before that meeting to read the other proposals of interest. In this meeting, the highest ranking proposals are put to special scrutiny, because there could be objections by other evaluators who had been critical in their reviews. Important is that the same standard is applied in the overall scoring for each proposal.

I think that this process works quite well and is very well taking into account the difficulties in achieving a fair scoring. I noticed that over the past 10 years the scoring had been somewhat simplified: in 2000 there were 5 criteria, with different weighting. The latest evaluation I attended, only had 3 criteria, with equal weighting.

Making this Blog Public

After more than 1/2 year of rest, I want to re-open this blog and use it to keep a reference of my activities. I feel that it would be useful to share this publicly, and therefore I have decided to make this blog available to the general public, without any password restriction. Confidential information which I had posted here earlier for the originally limited reader community has now been removed.

I hope to give updates on my work activities at Leeds Met, which can inform those who are interested in my work and projects.

This blog is closed for now

I have began using a Google Group for the updates. Google group provides a much easier way of sharing my entries to a limited number of readers: the blog here could not be read from a blog reader, due to password constrictions, whereas a Google group allows customisation of each individual member's preferences for getting updates (per email, or via web).

In this blog there will be no entries anymore from now on.

Bye!

Planning the Virtual Runner Project

After sending an email to the Google Group of our Leeds Met Centre for Creative Technology about the successful bid to Connect Yorkshire for the Virtual Runner Project, I have received several responses:

• Duncan Mullier would be interested in working on software development of this project. He would participate in writing core components.
• Mark Dixon has expressed interest in the software development activities.
• Patrick Ingham has been suggested to be involved, because of his dual expertise in software development and computer graphics.
• L3 students Abraham Smith has been suggested by Duncan, and I spoke to him: he would be interested, but feels that this may be slightly above his expertise.

Furthermore, I met with Simon Thompson who needs funding for purchasing Apple IPhones for some related development activities. I will explore if this VR project can support his exploration of using IPhones for an initial component development.