Force benchmarks to a specific set of parameters

Zama FHE Libraries Technical Support TFHE-rs library
1

Hey, I’m trying to run the unsigned 64bit integer throughput bench on cpu for a certain set of parameters

__TFHE_RS_PARAM_TYPE=classical _TFHE_RS_BENCH_TYPE=throughput cargo bench --bench integer-bench --features=integer,internal-keycache,nightly-avx512,pbs-stats -p tfhe-benchmark --

or editing the root makefile:

BENCH_TYPE?=throughput
BENCH_PARAM_TYPE?=classical
BENCH_PARAMS_SET?=default

and then running: make bench_integer

The default parameters for the bench are PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M128. I want to force this benchmark to its 2^-64 pfail variant, V1_1_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M64.

Edited tfhe-benchmark/src/param_alias.rs to add those parameters:

// KS PBS TUniform
    pub const BENCH_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M64: ClassicPBSParameters =
        V1_1_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M64;

Edited line 449 to be those default parameters on cpu for integer params in tfhe-benchmark/src/params.rs:

let params = vec![BENCH_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M64.into()];

however even after cleaning the project it doesnt seem to recognize the change and still uses the original V1_3_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M128 benchmarks.

If you could point me in the right direction how to change the parameter set that would be great.

2

Your approach looks good to me. Could you tell me on which commit SHA is based your development branch ? So that I can pinpoint the exact spot where the issue arise.

3

Thanks.

The git commit is 09aaa4e04510402f0ebfbdfb1cff138fd2598f50

4

From what I’ve tested, it’s expected to have a V1_3_ version of this parameter since this parameters set is the same since V1_1_. We use alias throughout versions to avoid declaring all parameters each time.
See the file tfhe/src/shortint/parameters/v1_3/classic/tuniform/p_fail_2_minus_64/ks_pbs.rs, you’ll see the following lines:

pub const V1_3_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M64: ClassicPBSParameters =
    crate::shortint::parameters::v1_1::V1_1_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M64;

The reason why you see V1_3_* parameters name instead of V1_1_* is due to the way we generate names via a macro.
In any case it should be a 2*-64 parameters set not a 2*-128 in your case.
I know that criterion tends to chop very long names, often we can read the beginning of the parameters name but not the P-fail associated just like:

Benchmarking integer::add_parallelized/integer::add_parallelized::V1_3_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M...: Warming up for 3.0000 s

Is this your case ?

5

ohhh ok that makes sense, thanks for that :slight_smile:

6

Is it working as you expect then ?

7

Yeah I’ve confirmed that it uses the 2^-64 pfail variant. The cargo bench outputs a directory benchmark_parameters , if you look inside one of the json files it shows the parameter set used which its using:

{"display_name":"negation","crypto_parameters_alias":"V1_3_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M64","crypto_parameters":{"lwe_dimension":879,"glwe_dimension":1,"packing_ks_glwe_dimension":null,"polynomial_size":2048,"packing_ks_polynomial_size":null,"lwe_noise_distribution":"TUniform(46)","glwe_noise_distribution":"TUniform(17)","packing_ks_key_noise_distribution":null,"pbs_base_log":23,"pbs_level":1,"ks_base_log":3,"ks_level":5,"pfks_level":null,"pfks_base_log":null,"pfks_std_dev":null,"cbs_level":null,"cbs_base_log":null,"br_level":null,"br_base_log":null,"packing_ks_level":null,"packing_ks_base_log":null,"message_modulus":4,"carry_modulus":4,"ciphertext_modulus":{"modulus":0,"scalar_bits":64},"lwe_per_glwe":null,"storage_log_modulus":null},"message_modulus":4,"carry_modulus":4,"ciphertext_modulus":64,"bit_size":8,"polynomial_multiplication":"Fft","precision":2,"error_probability":4.547473508864641e-13,"integer_representation":"Radix","decomposition_basis":[2,2,2,2],"pbs_algorithm":null,"execution_type":"Parallel","key_set_type":"Single","operand_type":"CipherText","operator_type":"Atomic"}

Thanks.

8

Glad that your issue is sorted out.

Regards.

9

Incase anyone or myself needs come back to this at some point I had to update:

// KS PBS TUniform
    pub const BENCH_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M64: ClassicPBSParameters =
        V1_1_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M64;

to

// KS PBS TUniform
    pub const BENCH_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M64: ClassicPBSParameters =
        V1_3_PARAM_MESSAGE_2_CARRY_2_KS_PBS_TUNIFORM_2M64;```

and it worked.