CCC supported three scientific sessions at this year’s AAAS Annual Conference, and in case you weren’t able to attend in person, we are recapping each session. This week, we are summarizing the highlights of the session, “How Big Trends in Computing are Shaping Science.” In Part 5, we share the Q&A portion of the panel. The panel comprised Jayson Lynch (Massachusetts Institute of Technology), Gabriel Manso (Massachusetts Institute of Technology), and Mehmet Belviranli (Colorado School of Mines), and was moderated by Neil Thompson (Massachusetts Institute of Technology).
Neil Thompson: “Gabriel, you highlighted just how dramatic the escalation in the amount of computing being used by these models is, and some ways we can mitigate this trend. How optimistic are you that we will be able to bring that level down enough that some scientists in our audience could utilize these systems in the future?”
Gabriel Manso: “I feel very optimistic, but I am not 100% sure. When we are talking about Moore’s Law, algorithmic improvements, and gains we can achieve from specialized hardware, I am more optimistic about the algorithmic side of things, but all of these areas need to be explored more in depth. I’d like to also mention that most of these state-of-the-art models that we currently use are transformer based models, which are heavily dependent on parallelism and computation. As an alternative, we should also begin to think about different possible architectures that would use fewer computing resources.”
Neil Thompson: “To continue on this question, Jayson, you mentioned parallel and approximate algorithms; how much potential do these algorithms have to reduce the amount of compute needed to run current models?”
Jayson Lynch: “As we saw for exact serial algorithms, we’ve reached the end of optimization for a lot of these problems. However, there is a lot of room left in improving approximation algorithms. We’re currently working on measuring how much room for improvement is left, so hopefully I will have some more precise measurements for you in a year or two. Specifically for deep learning, it is very difficult to predict development in one particular field. For some of the case studies that have looked at other problems that are in this highly-parallel approximate regime, we have seen steady exponential improvements over quite a few years. So I am reasonably optimistic that we are going to continue seeing algorithmic improvement in areas of deep learning. Some studies have shown the effective compute rate is doubling every nine months to two years depending on which area of Machine Learning you are measuring for. I bet these trends will continue, but are we going to see one thousand or one million fold increases in performance? Or will this occur over multiple decades? I’m really not sure.”
Neil Thompson: “Mehmet, I’ll ask you the same question. Specifically in deep learning, how do you think hardware will keep improving?”
Mehmet Belviranli: “The entire focus of the hardware industry is now on how to make AI faster, which is good news, specifically for deep learning. Cerebras, a hardware company working in this space, just released the fastest AI chip in the world. Developments like these are very promising, because these new tiny chips can have the ability of 40 GPUs and take up much less space. The space is looking very exciting.”
Neil Thompson: “Jayson, when a lot of people think about the end of Moore’s law, people think, ‘Oh, well quantum computers will come along, so it isn’t a big deal.’ That is a little deceptive because quantum computers run much more slowly than classical computers, but can run different algorithms. So what do you see as the potential for quantum algorithms?”
Jayson Lynch: “In very specific fields, quantum computers could have a huge amount of benefit, but they are not going to be a general purpose solution to all of these problems. Quantum computers give us different fundamental operations, and sometimes give us substantially more efficient algorithms, such as exponential speed-up for factoring, at the risk of breaking RSA encryption. That’s a case where dropping from exponential to polynomial time is such a large algorithmic increase, that once we have sufficiently large quantum computers, they will almost certainly outperform classical computers on these operations, regardless of the large slowdowns and difficulties associated with making these quantum gates. But, for the vast majority of problems, we don’t see speed-ups that are as large. So there will be really large problem size domains before we actually realize effective speed ups from these algorithmic gains.”
Audience Member: I wanted to know in your models of diverse computing, is there a place for analog logic? If you think about it, it takes thousands of transistors to add two floating-point numbers, but adding two voltages together takes one transistor. On the other hand, we have noise and temperature issues, but those don’t matter for certain kinds of application domains. So does your model of diverse computing include a sort of ‘throwback’ to the days of analog computing?
Gabriel Manso: “I can partially answer that question. I know there are big programs at DARPA that are just focusing on random number generation that uses analog logic. There are also several programs that focus on temporal domain logic. There is an interest, but the interest only goes so far as funding agencies support it.”
Audience Member: “I appreciate the discussion here in focusing on the opportunities available and the future of heterogeneous hardware and algorithms. However, the issue I think is of the most importance in terms of actually achieving those gains is less focused on the hardware and more focused on the people programming the software to run on the hardware. Also, facilitating cultural shifts in science towards prioritizing the need for researchers to have access to efficient algorithms in an open and accessible way that is intuitive to scientists who are not necessarily software engineers. So, how do you see us, as a community, better being able to help researchers within and outside of computational science use these resources and understand how to use the most cutting edge algorithms and hardware?”
Mehmet Belviranli: “These programming abstractions are incredibly important. There is an organization called Khronos, an association of many industry leaders, and they recently came up with an extension of Open-CL, which provides a C++ 17 compatible application, and the runtime backend is provided by the hardware vendor. The mechanism claims that you will only have to program your software once, and the hardware vendor is supposed to run your program in the most performance-efficient way, regardless of the hardware composition. So there are efforts in this space, and the tech companies are aware of that. DoE is heavily supporting research to make programming easier as well. The more vendors invest in such unified efforts, the better our future will be. Otherwise, even if our hardware continues to improve, if you can’t easily program on it, our gains won’t really matter for the majority of the scientific community.”
Neil Thompson: “Let me build on that for a second. We’re here at AAAS, and many people here are very concerned about this question of, ‘Well, if I need to use more computing, how do I do that?’. I’m curious, how much do you think that any scientist who came today, if they want to speed up their computation, how likely is it that they will have an ASIC that will actually help them? Because many of these chips are designed for very specific mathematical features.”
Mehmet Belviranli: “Building an ASIC is a very costly business, but there are efforts that make coding FPGAs much easier. Many vendors provide compilers, so you can write simple C code, and it will become decently efficient FPGA execution logic. There is certainly a learning curve to doing this, but it is much easier than it was 10 years ago.”
Audience Member: “I’d like to introduce another important aspect, which is edge computing and the need for high performance computing. An example of this is autonomous or ‘assisted-driving’ vehicles. Often the vehicle signal is lost, so you need to have computing power in the vehicle itself to do this. Also, how do we help the algorithms and hardware to be able to have computation to enable self-driving vehicles? Also, another big area is the interplay between solar power and battery storage. You need very complex compute and weather prediction systems to estimate when to use solar power from a commercial building or a large solar farm, and whether to send that electricity to the grid or store it in batteries. So does the panel have any thoughts on any of these issues?”
Jayson Lynch: “Those are both great examples of interesting areas of compute. In edge computing, there are many areas where you are extremely power limited, like in space, biomedical, and remote sensing applications. So there are lots of areas where we do need to make fast decisions or have heavy amounts of compute, and do both those things in an energy efficient manner. The advances in hardware and Moore’s Law are going to help with that. This is definitely a use case for specialized hardware; engineers working on autonomous drones are currently grappling with many of these problems. There have also been lots of algorithmic advances. We can do a lot with deep learning and vision in impressive ways at the moment, but can you create very small or very efficient neural networks that have sufficient performance? Especially if you put constraints on what your environment will look like, where you don’t need as many general purpose answers. Those are interesting deep algorithmic questions. Also, with optimizing the grid, this is something that is almost certainly an NP hard, very difficult optimization problem, but there is the question of, ‘How good of an answer do we need?’ and ‘Can we break this into several problems, or do we have to compute it at scale?’. If a lot of this computation needs to be done regionally, rather than at a country or global level, that makes your problem size much smaller. If you don’t mind having an answer that may overshoot your batteries by, say, a factor of two, it might be far easier to compute what a reasonable distribution is, as long as we are willing to spend some extra resources on building buffers for when these predictions are not accurate.
Neil Thompson: “The question of what is happening on the edge and in cars is a very interesting question because, what Gabriel showed is that so much of our performance improvement is coming from scaling up these deep learning systems. If you say I want to have great computer vision and scale up these systems, if it takes more energy, that is a big problem for your car. That decreases your mileage, you have to carry more GPUs, and you end up with real challenges. So that is a case where the algorithmic side of things is under-explored, and is probably the most important dimension. In some of the work my lab has done, we can see very rapid improvement in deep learning algorithms as well, and the question is how quickly will that bring down those costs so you can put a more powerful and more effective system in your car.
Gabriel Manso: “My recent research has mainly centered on autonomous vehicles and edge computing. If you have a drone running object detection algorithms, your drone quickly runs out of battery and you now have an energy budget but want to continue with a decent amount of computation. How would you use the two accelerators you have in the system more efficiently to do that? It is a very long road, especially with the programmability of the edge devices we have that are very domain specific. It is challenging, but people are definitely aware that you should not be using edge devices as intended, doing one computation at a time, because the computation is more demanding now, and these devices need to run multiple algorithms at a given time.
Thank you so much for reading! This was the final CCC at AAAS 2024 recap, and we hope to bring you more panels at AAAS 2025 in Boston.
