Hi, I am new to Concrete ML. I currently try to build an encrypted neural network. I found that the inference time difference between “execute” and “simulate” modes is pretty big. In my case and my device, a single image inference in “simulate” is less than 1s, while “execute” will take around 20 minutes.
First, I want to make sure. Is that big difference a normal thing in Concrete-ML? I tried the VGG example, “simulate” mode gives me around 40s inference, while the “execute” mode takes more than 20 minutes (I don’t know the exact number, as I need to stop it)
Second, as far as I know, in the deployment, I should use “execute”. But what exactly is missing in “simulate” mode? When should I use “execute”? For example, if I test adversarial attacks by analyzing the encrypted weight and so on in “simulate”, does it still represent in “execute” mode
Yes — it’s completely normal to observe a large runtime gap between fhe="simulate" and fhe="execute" in Concrete ML.
fhe="execute" runs real FHE crypto operations, which is much slower and mandatory in deployment scenarios (you won’t deploy an application if it’s not real FHE operations).
fhe="simulate", as its name suggests, it is a simulation, ie, it does not include real FHE operation, that’s why it’s faster, the purpose of this mode is to estimate the performance of the model given the crypto-parameters / FHE contrains / the noise / the FHE circuits … etc, which is really helpful in development scenarios.
How to make your model faster in execute mode?
Nowadays, working in FHE (or any other cryptographic paradigm) comes with constraints, so you often need to relax the constraints. For example, you can tune high-level cryptographic parameters such as p-error or rounding_threshold_bits, you can also use pruning.
Thank you for your reply. If I understand correctly, in “simulate” the data is not really encrypted. So let’s say I do plaintext-ciphertext multiplication P_A \times C_B. Then I want to do something like a statistical analysis during FHE computation, for both P_A and C_B.
Then, in “simulate” mode, the P_A analysis should be similar (or identical) when I do it in the “execute” mode, while the statistical analysis of the C_B may not represent the same result with the “execute” mode. Am I correct?
Your operation will be mapped to a computation FHE graph.
The simulate mode, uses the same constraints as the execute mode, ie same crypto params, same p-fail (error probability to get a bad result, which is very very low), it evaluates the bit-width of intermediate values to ensure the computation is valid under the chosen crypto parameters, etc.
At the end of the process, simulate and execute will yield the same result (p-fail guarantees it).
However, if you want to deep dive and analyze intermediate values , I recommend you to ask the Concrete channel of this forum, as they handle the low-level cryptographic implementation. They can guide you in displaying the statistics of the calculation graph.