Blogger: Alfonso Rodriguez-Molares, Postdoc and Senior Engineer, Department of Circulation and Medical Imaging and Centre for Innovative Ultrasound Solutions (CIUS).
A revolt is being stirred up in the beamforming world, and CIUS is among the insurgents.
For the first time in history a beamforming challenge was organized in the field of Medical Ultrasonics.
“What is beamforming?”, I hear you say. Beamforming is what bats do. They send an ultrasound signal which gets reflected at targets in front of them, such as prey (insects, fruits) or obstacles (watch out for that tree!). The reflected signal comes back to the bat ears and then the bat knows where the targets are and how to move around. But how does the bat know that? That’s where beamforming comes into play. Beamforming is the process by which temporal data (what the bat hears) is converted into spatial maps (where things are in the world). The bat does it with specialized ears that filter sound differently depending on the direction: up sounds differently than down, left than right, close than far. The same trick is used by other animals. Dolphins use an acoustic lens to direct ultrasound in the forward direction, oilbirds use echolocation to navigate through tress and caves in complete darkness, blind people can learn to navigate around the world by clicking their tongues. All that’s beamforming.
In ultrasound imaging we use a slightly different trick. Contrary to nature, we are not limited in the number of “ears” we can use. Ultrasound probes contains hundreds of elements (even up to a thousand in 3D probes) and each element is able to “hear” ultrasound. Then, just by using good old geometry (thanks Pythagoras!) we can estimate where the targets are and build ultrasound images. The more elements we have, the more accurate the image becomes, and that’s why we ended up using so many elements.
Ok, back to topic. In a beamforming challenge research groups from all over the world compete to generate the best possible image from a given ultrasound dataset (the set of “sounds” received by each element). Contrary to image processing challenges, which are common nowadays, this challenge focused only on beamforming algorithms.
Many sophisticated beamforming algorithms have been proposed in recent years. Among them adaptive beamformers have broken the rules of what we though feasible, obtaining a resolution and contrast never heard before. However, the new approaches are rarely compared against each other or against a common dataset. That makes it difficult (if not impossible) to assess how relevant these new techniques are, or if the increased quality claims can truly be substantiated. The ultrasound community feels a bit in the dark when it comes to adaptive beamformers.
The beam forming challenge PICMUS is a way to address this issue. By providing the same dataset to all participants and using the same metrics to asses image quality it is possible to objectively compare the performance of the algorithms. Points are given for each metric so that the participants can be ranked for each category. The top algorithms are then declared winners of their respective category.
In such a challenge, who the winner is, actually becomes secondary. The real value of the challenge is what we learn from the comparison and the nature of adaptive beamforming algorithms. It seems that comparing beamforming algorithms is not as easy as it may sound. There is not such a clear boundary between image processing and beamforming techniques, and different people may have different opinions on what can or cannot be done as part of a “beamforming” algorithm. In addition we detected several flaws in the metrics which could be “exploited” to achieve unrealistic values.
In this sense the outcome of PICMUS became invaluable. We are now on the track to defining better metrics that will allow us to eliminate these flaws. We understood the need for publicly available datasets, datasets in a common format that everybody can download and test their algorithm against. These two things together may help us change the way we look at beamforming results, allow us to detect unsubstantiated claims and separate the wheat from the chaff.
By achieving this, we can cut the time spent in fruitless discussions, shorten lengthy reviewing processes, and concentrate on the ideas that really make a difference.