Dr. Mark Read

Research Fellow, Charles Perkins Centre, The University of Sydney
Research Gate
My Research Gate

Paper on Calibrating Complex Biological Simulations Released!

RSInterfaceIt’s publishing season. Like the buses, nothing comes along for a while, and then all the papers are published at once. Another grand effort that has been 3 years in the making (since I started at the University of Sydney), the Journal of the Royal Society Interface this week released our manuscript on automated calibration of complex biological simulations. The original idea was formed near the end of my PhD, and is part of a larger theme on how to engineer accurate and representative simulations of biological systems that are incompletely understood: how do you simulate something if you don’t know how it works?

The answer in part rests on “calibration“, a process whereby you adjust the simulation such that its output matches that of known reality. Typically this involves finding parameter values, where the biological correlate is unknown. There are any number of approaches for doing this; the trick for complex system simulations is how you measure that difference between simulation and reality. Being complex, these systems cannot be well characterised in single observations or metrics alone, and that immediately blows standard techniques out of the water. An example, in this paper we employ the “ARTIMMUS” simulation as a test case, it simulates mouse multiple sclerosis. There are four chief T cell populations involved in the disease and subsequent recovery stages of this disease, each growing in population size, peaking, and falling again in a unique manner. They are all critical, and you can’t calibrate on the basis of one alone. Now that’s quite lot level, why not just characterise “disease severity” instead? Well, for starters “disease” is a very emergent property that is hard to replicate in simulation, we tend to deal with more concrete measures that can be tied to specific phenomenon. Take multiple sclerosis, there’s any number of ways that different areas of the nervous system can be impacted to deliver a given degree of debilitation.

mocEnter our approach. We use multi-objective optimisation to simultaneously evaluate several metrics of simulation’s capture of important biological features simultaneously, and evaluate find appropriate parameter values accordingly. We name this technique Multi-Objective Calibration (MOC). MOC exposes where several parameters may trade off against one another to deliver a given simulation dynamic, or where certain aspects of the simulation dynamic are maximised at the expense of others. This raises another intriguing possibility: what does it mean if no parameter values can be found that simultaneously align all aspects of a simulation’s output with that of reality? We propose that this points to a simulation that fails to adequately capture the complexities of the biological components; the changes that need to be made are not parameter values, but rather what those parameters represent. Perhaps an important cell population or component in the biology is missing from the simulation, or has been incorrectly captured. In this manner, we propose that MOC can play a vital role in guiding simulation design and development, not only parameter tuning at the end.

Sydney Bioinformaticians Get Together

Towards the end of one excellent conference (ANZOS 2014) I saw the call for another workshop, and so threw in an abstract. The abstract was accepted as a post-grad talk, and so I would be presenting at the Sydney Bioinformatics Research Symposium 2014. This was a smaller, though still well-attended affair hosted at the CPC.

There were some noteworthy highlights. Prof Andreas Zankl (a clinician) give a stunning talk littered with computer science, he and his team (of 3!) are designing a new database and user interface to collate together data and opinion on bone dysplasia. It was just so strange to have a GP tell me about the semantic web. As with last year, this year’s gathering had a “fast forward” hour wherein those with posters were given 2 minutes (exactly, you were cut short the second you overran) to draw attention to their work. It’s an adrenaline fuelled experience for the presenters, well aided by David Lovell’s considerable talent as an MC.

My own talk was on neutrophil swarming, and specifically how automated calibration can be integrated into the simulation development cycle to help refine the model. It was very well received, moreso than I think most other talks I’ve given thus far in my career. My approach in using simulation to test biological hypotheses differs from typical bioinformatics work, which is concerned with making sense of large quantities of high throughput data. I think both communities enjoyed learning a bit more about how the other’s approach works.

This year has not lacked for interesting conferences and workshops, though I’ve barely left Sydney. Its fantastic here, but I suppose the downside of living in an exotic place is that the conferences tend to come to you, rather than giving your the opportunity to head abroad.

Spartan released!

This week the “Spartan” paper was finally published in PLoS Computational Biology. Its always exciting to see the paper formatted in the journals own style. Its a shame about the layout of the two figures with captions falling on the following page – but never mind. The paper is first authored by one of Jon Timmis‘ past PhD students, Kieran Alden.

Spartan is an R package that encapsulates 4 key statistical analyses that have been of tremendous use in the computational immunology lab for understanding simulation results, revealing when these results are indicative of the biology rather than uncertainty regarding parameter assignments or the exact nature of the biological system itself. There’s a technique for determining how many samples of a stochastic simulation are required in an experiment to bring the effect of random variation down to a certain level of statistical significance (you want variation to be because of the experiment, not because you didn’t take enough samples). There is a robustness analysis that reveals how far simulation parameters can be perturbed before the simulation experiences a significant change in behaviour, and there are two global sensitivity analyses: eFast and a latin hypercube design with correlation measures.

If there’s a key message that has emerged from my own research, and echoed in other works coming from the lab, its that people probably trust simulation results a lot more than they should. The techniques gathered within spartan are a first step to understanding exactly how much you should believe the results coming from a simulation. I have a feeling I might make a career looking at this field, its important and very tricky.

PDF of paper, and the PLoS website.

SPARTAN paper accepted to PLoS Computational Biology

It’s been a while since I put anything on the blog. This doesn’t mean that I’ve not been doing anything – rather I’ve been doing so much that the blog gets neglected! There’ll likely be a flurry of activity in the run up to Christmas as I (and others) wind down.

Anyway, great news received last week. The SPARTAN paper has been accepted to PLoS Computational Biology. This is an excellent journal, so great cause for celebration. That can be added to the celebration that its first author, Kieran Alden, passed his PhD VIVA last week too.

SPARTAN is a collection of tools focused on sensitivity analysis, and understanding the link between simulations and biology. It provides (amongst other things) an implementation of the robustness analysis technique I published earlier this year. It also includes comprehensive tutorials for how to use the package, and it is already being used by others within Jon Timmis’s lab. PLoS Comp Bio is an open access journal, so I’ll provide links to the manuscript once its been typeset and placed online by the journal.

PS. We have also heard that the BMC Bioinformatics paper should be in print in February 2013.

Journal paper accepted to BioSystems

We have just had word that our paper, first authored by Richard Greaves, has been accepted for publications in BioSystems. It is an extension of the conference paper presented at IPCAT this year, firth authored by Richard Greaves, and includes work that Bjorn and James did on modelling CD200 regulation of dendritic cells in EAE. Bjorn and James were placement students from Leeds university who are both starting PhDs under Jon Timmis (and others). It is truly remarkable that the placement students’ work would end up in a journal, and a testament to their potential.

The paper examines several possible mechanistic models of how CD200 negative signalling of DCs suppresses their ability to promote autoimmune T cell responses, and clearly motivates further work in the computational immunology field on principled approaches to designing and building simulations. As luck would have it, this is work that is on our horizons.

ICARIS in progress!

This year’s ICARIS conference (international conference on artificial immune systems) is held in Taormina, Sicily. The university of York has 6 delegates here, covering both the computational immunology and engineering concerns of the conference.

Richard Williams’ had the “grave yard” shift, presenting his position on the limitations of UML in capturing the stochastic aspects of the immune system, and how he has used statistics to model the distributions of immune behaviour that result from this stochasticity. Jon Timmis presented on day 2, arguing that if immunological simulations are to inform drug discovery and design, and clinical trial design as we all hope, then they must be viewed as safety critical systems (the safety of patients depending on this information). As as result, the processes through which they are constructed, tested for bugs, documented and demonstrated as reliable must all become more rigorous. This is a message that has been emanating from our lab for some time now (my own thesis addresses some of these issues), and its very encouraging to see the audience being receptive to the message. The same “grave yard” shift of day 2 (at the end of a long day, with a completely dark room) saw my own presentation on possible cellular mechanisms through which CD200 inhibits DC function (work performed by Bjorn and James). This too was well received, though I learnt that talking over a live video of ARTIMMUS that is already annotated with text can be tricky to time correctly!

We have just, literally just now, had Tiong’s paper on sensor networks. There was some comedy as Tiong tried to start his presentation on a Chinese chap’s computer – only to find that he could not full screen the presentation as all the menus were in Chinese symbols!

I leave you with this lovely picture of Taomina…

Update: the conference is now coming to a close. Its been fantastic to see colleagues in the field again, who have been very social – 18 of us went to dinner together last night, where the following photo was taken of the day’s plenary speakers, Prof. Stephanie Forrest. She delivered an outstanding presentation on “The Biology of Software”, where she described research on using evolutionary methods to fix bugs in computer code (amongst other things). She was very happy at dinner.

PS. There has been a conspicuous gap in my own publication record for the last two years, being the bulk of my research that resulted in the construction of the ARTIMMUS simulation. This is not dead, its just turning out to be an extremely hard paper to write given where we are trying to send it. Its on the 10th complete re-write in 2 years. We found some time to look at it again in the conference, and this draft is looking good, so hopefully there will be more news on that soon enough…

YCIL website live

The group researchers at York engaging in computational immunology have launched a lab website showcasing our, and are now proudly known as the York Computational Immunology Lab (YCIL). The lab has grown quickly over the last few years, and the number and breadth of research conducted is impressive: from multi-compartment systemic autoimmune diseases like EAE to molecular-level modelling of influential factors like NF-kB. Several new students have recently joined the lab, and we will shortly be adding Crohn’s disease to the list of areas studied. The lab researches not only diseases and systems as a complement to traditional techniques, but also develops technologies to support these activities. Simulations of biological systems can encompass a great many parameters, and the lab is examining how cutting edge technologies can be used to meet the substantial computational demands of calibrating and executing these simulations. I look forward to writing more about the excellent work being conducted in the coming months. For now, have a scan of the website.

Manuscript accepted by BMC Bioinformatics

During their respective Masters degrees, Richard Williams and Richard Greaves conducted some very elegant experiments using the ARTIMMUS simulation that elucidated the nature of regulation in EAE. Richard Williams’ work won the Best Immunological Modelling Paper award at ICARIS 2011, and it was his abstract that lead to the current journal submission. Richard W’s work was combined with work from Richard G, and yesterday the manuscript was accepted for publication in the BMC Bioinformatics journal. Winning a best paper prize and then being accepted for publication in this very respectable journal is a fantastic testament to the very high quality of work that these two have conducted (as Master’s students, no less)!

The paper reports three experiments, all excellent exemplars of how simulation facilitates work that could not be conducted in vivo. Manipulation of dendritic cell peptide presentation demonstrates that the regulation mediating recovery from autoimmunity in this model of EAE does not require regulatory T cells and their autoimmunity-inducing targets to be primed by the same dendritic cells. The paper reports that although CD4Th cells are invaluable in facilitating the cytotoxic T cell response, their physical presence and the temporal aspects of signals they deliver to dendritic cells modulate the magnitude of this response. Artificial means to deliver the signals that CD4Th cells are responsible for could yield more aggressive CTL responses that might overcome otherwise persisting disease.

Electronics departmental seminar

I have in the past presented at conferences, special interest and research groups, but yesterday was my first departmental seminar. There was some initial concern about low numbers – the suitability of speaking to electronics researchers about simulating the immune system had previously been raised, but I felt that the motivation, potential impact and issues of computational immunology could be conveyed without excess immunological detail. Many in Electronics use simulation, and there was interest in simulating systems that are themselves not well understood.

The drive of my seminar was to demonstrate how computational simulation could aid drug discovery (which is challenging and very expensive), but that simulation results are not necessarily representative of the real immune system, and to highlight the techniques I employ to build confidence that they are. I’m pleased to say it was very well received. The next might actually be on swarm robotics!

CD200-modelling abstract accepted to ICARIS 2012

I received the excellent news that an abstract concerning the modelling of CD200, a molecule implicated in modulating dendritic cell activity and recovery from EAE, has been accepted as an oral presentation at this year’s ICARIS conference. This work was conducted by Bjorn and James over the Christmas holiday – both are starting PhDs with Jon Timmis in October, we have high hopes for them both!