Accelerating Homomorphic Encryption Based Privacy-Preserving Machine Learning

With machine learning becoming ubiquitous in today’s enterprises and software platforms, solutions must incorporate privacy-preserving techniques across artificial intelligence (AI) platforms. While this idea might seem obvious today, AI research communities historically focused on breaking boundaries across data silos and matching data points from one silo to another, gleaning previously unnoticed insights.

As the field of AI has matured, it has become evident that without protecting private data, we can expose our data sources to potentially harmful exploits, leaving practitioners of AI responsible for the consequences. Today the AI industry has responded to this problem by working closely with cryptographers to address this critical juncture in AI technology. One methodology for keeping private data undisclosed that allows for inferring critical insights is called Homomorphic Encryption.

What Is Homomorphic Encryption? 

Homomorphic Encryption (HE) belongs to a class of advanced cryptographic techniques for privacy-preserving computation. It enables computation on encrypted data without ever decrypting it, only allowing an authorized party to decrypt the computation results. This unique cryptographic technique allows data to remain encrypted at rest, in transit, and during computation. Arbitrary computation on encrypted data had been sought since the late 1970s, until 2009, when Craig Gentry¹ described the first construction of a Fully Homomorphic Encryption (FHE) scheme. This breakthrough finally made it possible to perform arbitrary computations.

HE can be grouped into three broad types:

Fully homomorphic encryption (FHE): Described first by Craig Gentry in 2009, FHE is a type of encryption that supports both additions and multiplications within the same scheme and allows for generic computation of arbitrary depth to be performed on encrypted data without ever decrypting it. Popular examples of FHE schemes include BGV²/ BFV^3,4 / CKKS⁵ variants with bootstrapping, FHEW⁶, and TFHE⁷. FHE schemes are in active research and development and undergoing the process of standardization.

Fully Levelled homomorphic encryption (FLHE): Also referred to as Levelled Homomorphic Encryption (LHE), it is similar to FHE but more restrictive as it allows limited (or predetermined) computation depth. Popular examples include BGV, BFV, and CKKS without bootstrapping. These scheme variants are also under the scrutiny of standardization.

Partial homomorphic encryption (PHE): This form of encryption has existed for many years and allows either addition or multiplications (but not both) over encrypted data without ever decrypting it. Popular examples of PHE are the RSA⁸, the Paillier Cryptosystem⁹, and ElGamal Encryption¹⁰ schemes. These three schemes are standardized and commonly used in today’s production environments.

Starting Small with a Partially Homomorphic Solution 

FHE is considered the holy grail of cryptography; the data remains encrypted throughout its lifecycle, at rest, in transit, and when it is computed on. At the time of its formalization in 2009, the computational overhead was considered too slow for any practical use. FHE still lacks standardization and performance for generic use, and many users are conservative in implementing real-world applications based on HE. Advances in the last decade have started to realize the potential for homomorphic encryption to become a reality, particularly in the regulated industries and those where preserving the privacy and confidentiality of the data is paramount. 

Times have changed, and bad actors are more sophisticated. As a result, we need more secure applications. Now, through a collaboration with Intel, we have addressed the performance gap with a hardware-accelerated implementation of a partial homomorphic encryption scheme to better meet market demands.

Intel engineers developed the “Intel Paillier Cryptosystem Library” (IPCL)¹², the first open-source, ISO-compliant Paillier cryptosystem software implementation. IPCL is optimized on 3rd Generation Intel® Xeon® Scalable processors by leveraging Advanced Vector Extensions 512 (AVX512) and Integer Fused Multiply Accumulate (IFMA) features. 

IPCL is an important step towards standardization of security with privacy protection of federated learning solutions that use the Paillier Cryptosystem to meet data privacy regulations during computation with high computing performance. Intel partnered with VMware to deliver both the essential technology and the packaging and management to make it a complete, readily deployable solution. This partnership helps VMware build better products and offerings based on HE.

Experimenting with IPCL in KubeFATE

The Advanced Technology Group (ATG) within the VMware Office of the CTO developed KubeFATE¹³, an enterprise-managed solution that builds federated learning on Kubernetes for data centers. It is part of the open-source project Federated AI Technology Enabler (FATE)¹⁴, which is hosted by Linux Foundation AI & Data, to orchestrate infrastructure and service across organizations. 

FATE implements secure computation protocols based on HE and multi-party computation (MPC). Through the integration of IPCL with FATE, KubeFATE benefits from performance gains when it runs on the latest Intel processors. The key mathematical functions in IPCL leverage Intel AVX512 for SIMD parallelism and the Integer Fused Multiply-Add (IFMA) instruction set for faster processing time. More details about acceleration can be found in this previous article ¹⁵.

Figure 1 illustrates more details of the software stack with the integration of IPCL into FATE. This new feature is planned for FATE v1.9 release. IPCL in Python is provided as a bridge to facilitate the integration of IPCL to a Python-based framework.

Looking to the near future

Intel and VMware, together, have made enormous strides in modernizing the U.S. power grid by collaborating on virtualizing power substations¹⁶. The next requirement in this modernization process is incorporating next-generation encryption schemes such as HE and postquantum ciphers. 

While the electrical grid is a natural fit for this modernization effort, our water and sewer systems are also prime candidates. The maintainers of this critical infrastructure are also looking for ways to comply with new security and privacy legislation such as the California Consumer Privacy Act (CCPA)¹⁷, European Union General Data Protection Regulation (GDPR)^18, and White House Executive Order 14028¹⁹. 

Beyond public utilities, multi-cloud enterprises are looking for ways to adopt these technologies to keep their most business-critical data safe. According to analyst firm PwC, these same companies are actively looking for opportunities to utilize AI solutions to analyze these sensitive data sets to make faster business decisions and gain new insights²⁰. Gartner has listed privacy-enhancing computation as the number three most important strategic trend of 2022²¹:

“Privacy-enhancing computation secures the processing of personal data in untrusted environments — which is increasingly critical due to evolving privacy and data protection laws as well as growing consumer concerns. Privacy-enhancing computation utilizes various privacy-protection techniques to allow value to be extracted from data while still meeting compliance requirements.”

We are at the onset of a journey to continuously explore the usage of HE in solutions, including partial HE and FHE. As next steps, we are building solutions in HE, which can be accelerated using other open-source libraries from Intel, such as HEXL and Intel HE Toolkit.  

---

Authored by Martin Stack, Staff Designer, Office of the CTO, VMware

Special Contributions By: The Intel Private AI Team

• Anil Goteti, Business Development Manager, Private AI and Analytics, Intel Corp.
• Jingyi Jin, Principal Engineer, Director of Engineering, Private AI and Analytics, Intel Corp.
• Flavio Bergamaschi, Principal Engineer, Director and CTO, Private AI and Analytics, Intel Corp.
• Nir Peled, General Manager, Private AI and Analytics, Intel Corp.

Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries.

---

References:

1. Craig Gentry. Fully homomorphic encryption using ideal lattices. In Michael Mitzenmacher, editor, 41st ACM STOC, pages 169{178. ACM Press, May / June 2009
2. Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. (Leveled) fully homomorphic encryption without bootstrapping. In Shafi Goldwasser, editor, ITCS 2012, pages 309{325. ACM, January 2012.
3. Zvika Brakerski. Fully homomorphic encryption without modulus switching from classical GapSVP. In Reihaneh Safavi-Naini and Ran Canetti, editors, CRYPTO 2012, volume 7417 of LNCS, pages 868{886. Springer, Heidelberg, August 2012
4. Junfeng Fan and Frederik Vercauteren. Somewhat practical fully homomorphic encryption. Cryptology ePrint Archive, Report 2012/144, 2012. https://eprint.iacr.org/2012/144.
5. Jung Hee Cheon, Andrey Kim, Miran Kim, and Yong Soo Song. Homomorphic encryption for arithmetic of approximate numbers. In Tsuyoshi Takagi and Thomas Peyrin, editors, ASIACRYPT 2017, Part I, volume 10624 of LNCS, pages 409{437. Springer, Heidelberg, December 2017.
6. L´eo Ducas and Daniele Micciancio. FHEW: Bootstrapping homomorphic encryption in less than a second. In Elisabeth Oswald and Marc Fischlin, editors, EUROCRYPT 2015, Part I, volume 9056 of LNCS, pages 617{640. Springer, Heidelberg, April 2015
7. Ilaria Chillotti, Nicolas Gama, Mariya Georgieva, and Malika Izabach`ene. TFHE: Fast fully homomorphic encryption over the torus. Journal of Cryptology, 33(1):34{91, January 2020.
8. R. Rivest, A. Shamir, and L. Adleman. A method for obtaining digital signatures and public-key cryptosystems. Commun. ACM, 21:120{126, 1978.
9. Pascal Paillier. Public-key cryptosystems based on composite degree residuosity classes. In Jacques Stern, editor, Advances in Cryptology | EUROCRYPT ’99, pages 223{238, Berlin, Heidelberg, 1999. Springer Berlin Heidelberg.
10. Taher ElGamal. A public key cryptosystem and a signature scheme based on discrete logarithms. In George Robert Blakley and David Chaum, editors, Advances in Cryptology, pages 10{18, Berlin, Heidelberg, 1985. Springer Berlin Heidelberg.
11. Bootstrapping in Fully Homomorphic Encryption: https://www.boazbarak.org/cs127spring16/chap15_fhe
12. IPCL: https://github.com/intel/ipcl
13. KubeFATE: https://github.com/FederatedAI/KubeFATE
14. FATE (Federated AI Technology Enabler): https://github.com/FederatedAI/FATE
15. Accelerating Secure Compute for the FATE Framework: https://www.intel.com/content/www/us/en/developer/articles/technical/homomorphic-encryption/accelerating-secure-compute-for-fate-framework.html
16. https://www.businesswire.com/news/home/20220407005048/en/Crystal-Group-Advances-Energy-Modernization-With-VMware-Validated-Solutions
17. California Consumer Privacy Act (CCPA): https://oag.ca.gov/privacy/ccpa/
18. General Data Protection Regulation (GDPR): https://gdpr.eu/what-is-gdpr/
19. White House Executive Order 14028: https://www.nist.gov/itl/executive-order-14028-improving-nations-cybersecurity
20. https://www.pwc.com/us/en/tech-effect/ai-analytics/ai-business-survey.html
21. https://www.gartner.com/en/information-technology/insights/top-technology-trends