Dot Product using GpuFheXXArrays

Zama FHE Libraries Technical Support TFHE-rs library
1

Hello

I am trying to use the GpuFheUint16Array to type to perform matrix multiplication. My issue is with the dot product step, where I need to multiply two GpuFheUint16Arrays and sum the values. I can easily multiply the GpuFheUint16Arrays, but I cannot reduce the array into a single FheUint16 value (the sum). Is there any way to do that or convert theGpuFheUint16Array into a vector of FheUint16? Here is a part of my code (Ctxt = GpuFheUint16Array):

// Single CUDA kernel launch for all N element-wise products:
let prod_array: Ctxt = (&row_a * &col_b_vec);

// Reduce sum across the resulting array elements
let dot_product = FheUint16::sum(prod_array.iter());

I copied the following function from another question and changed it to GpuXX, but it fails to run:

fn uint_array_to_uint_vec<Id: FheUintId>(array: CpuFheUintArray<Id>) -> Vec<FheUint<Id>> {
    array
        .into_container()
        .into_iter()
        .map(|radix| FheUint::<Id>::try_from(radix))
        .collect::<Result<Vec<_>, _>>()
        .unwrap()
}

Thanks in advance!

2

Hello,

Can you be more specific in if fails to run what is the error ? who to you call the function ?

3

The error with the uint_array_to_uint_vec function is that into_container()does not yield an iterator.

error[E0599]: `tfhe::array::gpu::integers::GpuOwned<CudaUnsignedRadixCiphertext>` is not an iterator
  --> src/main.rs:36:10
   |
34 | /     array
35 | |         .into_container()
36 | |         .into_iter()
   | |         -^^^^^^^^^ `tfhe::array::gpu::integers::GpuOwned<CudaUnsignedRadixCiphertext>` is not an iterator
   | |_________|
   |
   |

.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tfhe-1.7.0/src/high_level_api/array/gpu/integers.rs:49:1
   |
49 |   pub struct GpuOwned<T>(Vec<T>);
   |   ---------------------- doesn't satisfy `_: IntoIterator` or `_: Iterator`
   |
   = note: the following trait bounds were not satisfied:
           `tfhe::array::gpu::integers::GpuOwned<CudaUnsignedRadixCiphertext>: Iterator`
           which is required by `tfhe::array::gpu::integers::GpuOwned<CudaUnsignedRadixCiphertext>: IntoIterator`
           `&tfhe::array::gpu::integers::GpuOwned<CudaUnsignedRadixCiphertext>: Iterator`
           which is required by `&tfhe::array::gpu::integers::GpuOwned<CudaUnsignedRadixCiphertext>: IntoIterator`
           `&mut tfhe::array::gpu::integers::GpuOwned<CudaUnsignedRadixCiphertext>: Iterator`
           which is required by `&mut tfhe::array::gpu::integers::GpuOwned<CudaUnsignedRadixCiphertext>: IntoIterator`

I think the main problem is that GpuFheXXArrays data structure does not implement the iterator trait, which prevents accessing individual elements of the array.

The same applies for the prod_array.iter(), the iter() function is not defined.

4

Sorry for the delay, I did not get a notificatin,

Yes you are a bit stuck, there is not convertion Array <=> Vec, in the released crate, and the GPU stuff is private so you can’t write your own convertion like you can for CPU

You can try to build with a custom branch that has these convertions


  [dependencies]
  tfhe = { git = "https://github.com/zama-ai/tfhe-rs.git", branch = "tm/array-from", features = ["integer", "gpu"] } 

  let arr = GpuFheUint32Array::from(vec_of_fheuint32);  
  let v: Vec<FheUint32> = arr.into();
5

Thank you so much for your consideration and effort!

I’ve not tried the this branch yet, but I will mark the question as solved.