A New Frontier in Machine Learning: from MLPs to KANs and Encrypted AI
**Author:** Valentina Kononova, Head of AI at Fair Math
Introduction
Artificial Intelligence (AI) has transformed almost every aspect of our lives, from how we interact with our phones to how scientific discoveries are made. At the core of this transformation lies a simple yet powerful idea: neural networks. These mathematical models learn to identify patterns and make predictions, enabling machines to "understand" language, vision, and data in ways that once seemed impossible.
But even as AI reaches new heights, researchers are beginning to explore new architectures that push beyond traditional neural networks. One such innovation is the Kolmogorov–Arnold Network (KAN), a novel architecture that may redefine how we build and interpret machine learning systems. Even more intriguingly, these architectures are being adapted to work with encrypted data, opening doors to a future where AI can be both powerful and private.
Let's explore how neural networks work, what makes KANs special, and how they're being combined with encryption to protect our data in the age of AI.
MLP: The Foundation of Modern AI
Modern neural networks, regardless of their architecture, are fundamentally built upon the concept of multilayer perceptrons (MLPs). At the core of many advanced models—whether it's convolutional neural networks (CNNs) or even the highly expressive transformer models—lie components known as fully connected or dense layers, which are essentially MLP blocks.
At a basic level, an MLP transforms input data using a combination of linear algebra and nonlinear functions. The typical operation looks like this:
f(x) = σ(W * X + B)
Where,
This combination of linear and non-linear operations gives neural networks their ability to learn complex patterns from data.
Over time, deep learning models have become deeper and more specialized, but the essential building block remains the same. Even transformers—arguably the most influential architecture of the last decade—rely on MLP layers to process and refine representations after attention mechanisms. The universality and simplicity of MLPs have made them a reliable choice for countless applications, from computer vision to natural language processing, reinforcement learning, and beyond.
However, this simplicity also comes at a cost. Despite their success, traditional MLPs are limited by the fact that they use fixed activation functions (AF) across all layers and neurons. Every node in a layer applies the same nonlinear transformation, regardless of the structure or type of data it processes. While this has worked well in many cases, it also means that the network has relatively rigid degrees of freedom when it comes to shaping its internal functions. In applications that require higher interpretability, structured reasoning, or fine-grained control over functional forms, MLPs may fall short or require overparameterization to compensate.
This limitation has sparked interest in exploring alternative architectures that could offer more flexibility and mathematical structure. One particularly promising approach revisits a classic result from functional analysis—the Kolmogorov–Arnold representation theorem [1, 2, 3]—and builds a new type of neural network architecture from it. These models are known as Kolmogorov–Arnold Networks, or simply KANs.
KANs: A promising alternative to MLPs
KANs are still relatively new in the machine learning world. Even though the theorem was proven almost 70 years ago, it was first demonstrated to be feasible in machine learning in 2024 by a team of researchers from multiple universities, led by Ziming Liu from MIT, in the article titled KAN: Kolmogorov–Arnold Networks [4]. While MLPs are based on the universal approximation theorem, KANs are inspired by the classical Kolmogorov-Arnold representation theorem. Liu modified this classical theory to extend the original two-layer design, making it possible to build deeper and wider networks.
Unlike traditional neural networks that rely on fixed activation functions, KANs use learnable univariate functions on each edge of the neural network. This means that instead of applying the same nonlinear transformation to all neurons in a layer, each connection between nodes can learn its own transformation. These learnable functions are typically represented using B-splines, allowing flexible and localized approximation.
The authors define activation function as a sum of basis and spline functions:
ϕ(x) = w_b · b(x) + w_s · spline(x)
where basis function b(x) = silu(x) = x / (1 + e^(-x)) can be used for most of cases, and spline(x) = Σ c_i B_i(x)
Liu's implementation demonstrated that KANs could outperform conventional multilayer perceptrons on a range of tasks, particularly those involving structured data or requiring high interpretability. The architecture also offers a more interpretable and mathematically grounded framework for understanding deep learning, as it ties back directly to theoretical results in functional analysis. The authors demonstrate that KANs outperform comparable MLPs on various toy examples and highlight their applicability in mathematics and theoretical physics.
FHE + KAN
MLPs serve as the backbone of many modern deep learning models, which is why most research in privacy-preserving machine learning has centered around them. However, integrating MLPs with homomorphic encryption (HE) presents significant challenges. A key issue lies in the activation functions commonly used in MLPs, which are nonlinear and cannot be directly evaluated over encrypted data. To work around this, many approaches approximate these functions using low-degree polynomials that are compatible with HE, as they require only addition and multiplication -operations inherently supported by HE schemes.
KANs incorporate additional nonlinear elements, which increase computational demands during training. However, the authors of the original paper demonstrated that it's possible to fix the activation functions (in the context of FHE, this may involve using polynomials, which have shown good accuracy) and leverage a hybrid approach combining MLPs with fixed activation functions and KANs with trainable ones at the edges [4].
As usual, when discussing privacy-preserving machine learning or integrating Fully Homomorphic Encryption (FHE), we need to consider two main stages: model training and inference.
Most existing research focuses on inference rather than training, which is understandable. KANs require more time to train in order to reach acceptable accuracy, and they also feature a diverse range of trainable nonlinear functions. For inference, you can use the symbolic formula derived from the trained KAN. In the context of FHE, these formulas can be adapted to enable evaluation on encrypted data.
The authors of "CryptoKANs: Enhancing Privacy-Preserving Machine Learning in IoT Environments using Kolmogorov-Arnold Networks over Encrypted Data" [5] build KAN supporting training and inference on encrypted data. First, they train the model on plain text to achieve acceptable accuracy, and then they convert each activation to a polynomial form to fine-tune the model on an encrypted subset, which allows CryptoKAN to outperform compatible MLPs. They chose to limit the polynomial degree to two in order to maintain a fair comparison with MLPs.
The authors of "Efficient Privacy-Preserving KAN Inference Using Homomorphic Encryption" [6] propose a novel method for approximating activation functions by dynamically selecting the approximation range and applying weighted least squares, resulting in an accurate polynomial approximation of the SiLU function suitable for homomorphic encryption. They also introduce an efficient approach for computing B-spline basis functions within the HE domain by leveraging repeat packing, lazy combination, and encrypted comparison techniques to minimize computational overhead. Experimental results demonstrate that the proposed HE-KAN model delivers strong performance on symbolic regression and image classification tasks, achieving significant speedup of inference on the CIFAR-10 dataset compared to a naive implementation.
FHERMA AI Challenges
FHERMA is a platform for FHE challenges with a focus on AI and blockchain. It provides a space for benchmarking machine learning models that operate on encrypted data. The platform hosts open challenges centered around privacy preserving inference and training. Challenges are regularly updated and reflect real-world use cases. The recent challenges include: CIFAR10, Ethereum Fraud Detection, House Price Prediction.
Challenge results are used to build an open-source library of FHE components called Polycircuit. It includes model architectures, activation functions, and supporting tools for developing practical privacy-preserving AI systems.
The KAN + FHE approach, described above, has been used by participants to solve challenges such as House Price Prediction, where both accuracy and data privacy are critical. This method combines the flexibility of Kolmogorov–Arnold Networks with encryption-compatible polynomial approximations, making it suitable for secure inference tasks.
FHERMA serves as a testing ground and collaboration hub for advancing encrypted AI — providing datasets, benchmarks, and reusable components to accelerate real-world adoption.
Conclusion
Artificial Intelligence is rapidly transforming the world around us — from healthcare and finance to communication, logistics, and scientific research. However, as AI systems become more powerful, they increasingly encounter critical challenges related to data privacy. In many applications, sharing raw data with AI models is simply not an option.
This is where Fully Homomorphic Encryption (FHE) offers a breakthrough. By enabling computation directly on encrypted data, FHE allows us to harness the full potential of Machine Learning without ever exposing sensitive information. It represents a key step toward building AI systems that are not only intelligent, but also trustworthy by design.
Combining KANs—a flexible and mathematically grounded neural network architecture—with FHE opens up new possibilities for privacy-preserving AI. This powerful combination is already proving effective in real-world tasks such as House Price Prediction and Ethereum Fraud Detection, showing that high accuracy and strong privacy can coexist.
The progress we see today is just the beginning. Platforms like FHERMA provide the space to test new ideas, share results, and build reusable tools for the encrypted AI ecosystem. We invite researchers, engineers, and enthusiasts to join the us, contribute to the community, and help push the boundaries of what secure AI can achieve.
Stay tuned — and be part of shaping the future of privacy-preserving intelligence.
References
1. Andrey Kolmogorov, "On the representation of continuous functions of several variables by superpositions of continuous functions of a smaller number of variables", Proceedings of the USSR Academy of Sciences, 108 (1956), pp. 179–182; English translation: Amer. Math. Soc. Transl., "17: Twelve Papers on Algebra and Real Functions" (1961), pp. 369–373.
2. Vladimir Arnold, "On the representation of continuous functions of three variables as superpositions of continuous functions of two variables", Dokl. Akad. Nauk. SSSR 114:4 (1957), pp. 679–681 (in Russian)
3. Andrey Kolmogorov, "On the representation of continuous functions of several variables as superpositions of continuous functions of one variable and addition", (1957); English translation: Amer. Math. Soc. Transl., "28: Sixteen Papers on Analysis" (1963)
4. Z. Liu, Y. Wang, S. Vaidya, F. Ruehle, J. Halverson, M. Soljačić, T. Y. Hou, M. Tegmark, "KAN: Kolmogorov–Arnold Networks", 2024, arXiv:2404.19756v5
5. O. Tahmi, C. Talhi, H. Ould-Slimane, "CryptoKANs: Enhancing Privacy-Preserving Machine Learning in IoT Environments using Kolmogorov-Arnold Networks over Encrypted Data", 2024
6. Z. Lai, Y. Zhou, P. Zheng, L. Chen, "Efficient Privacy-Preserving KAN Inference Using Homomorphic Encryption", 2024, arXiv:2409.07751v1