Today, we typically compute and store results in 64-bit double precision by default, even when very few significant digits are required. Because of the growing disparity of FLOPs to memory bandwidth in modern computer systems and the rise of General-Purpose GPU computing – which has better peak performance in half and single precision on some hardware – there has been renewed interest in computing in precisions other than double. The reality is that many of the bits in double precision are representing errors – truncation, iteration, and roundoff – instead of useful information about the solution. This over-allocation of resources is wasteful of power, bandwidth, storage, and FLOPs; we communicate and compute on many meaningless bits and do not take full advantage of the computer hardware we purchase.
It’s time to rethink how we represent and use real numbers with finite precision. Can we move beyond static, hand-crafted mixed precision implementations to more dynamic variable precision? Are there other representations that possess better properties than standard IEEE?
At LLNL, we are developing the methods and tools that will enable the routine use of dynamically adjustable precision at a per-bit level depending on the needs of the task at hand. Just as adaptive mesh resolution frameworks adapt spatial grid resolution to the needs of the underlying solution, our goal is to provide more or less precision as is needed locally. Acceptance from the community will require that we address three concerns: that we can ensure accuracy, ensure efficiency, and ensure ease of use in development, debugging, and application. In this talk, I will discuss the benefits and the challenges of variable precision computing, highlighting aspects of our ongoing research in data representations, numerical algorithms, and testing and development tools.
This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344 and funded by LLNL Laboratory Directed Research and Development, Project 17-SI-004.
Copyright. All Rights Reserved.