Random Initialization

Random and Weight Initialization

numc includes a fast thread-local PRNG (xoshiro256**) and common neural network weight initialization methods.

Seeding

numc_manual_seed

void numc_manual_seed(uint64_t seed);

Sets the seed for the thread-local PRNG. Call this before creating random arrays if you need reproducible results:

numc_manual_seed(42);
NumcArray *a = numc_array_rand(ctx, shape, 2, NUMC_DTYPE_FLOAT32);

numc_manual_seed(42);
NumcArray *b = numc_array_rand(ctx, shape, 2, NUMC_DTYPE_FLOAT32);
// a and b contain identical values

If you never call numc_manual_seed, a fixed default seed is used -- so results are deterministic but arbitrary.

Thread safety: Each thread has its own PRNG state. numc_manual_seed only affects the calling thread.

Random Array Creation

numc_array_rand -- Uniform Distribution

NumcArray *numc_array_rand(NumcCtx *ctx, const size_t *shape,
                           size_t dim, NumcDType dtype);

Creates an array filled with random values:

• Float types (FLOAT32, FLOAT64): uniform distribution in [0, 1), using an IEEE 754 bit trick for exact distribution.
• Integer types: full-range random bits masked to the type width.

// 3x3 random floats in [0, 1)
NumcArray *a = numc_array_rand(ctx, (size_t[]){3, 3}, 2, NUMC_DTYPE_FLOAT32);
numc_array_print(a);
// [[0.374540, 0.950714, 0.731994],
//  [0.598658, 0.156019, 0.155995],
//  [0.058084, 0.866176, 0.601115]]

// Random uint8 values (0-255)
NumcArray *b = numc_array_rand(ctx, (size_t[]){5}, 1, NUMC_DTYPE_UINT8);

numc_array_randn -- Normal Distribution

NumcArray *numc_array_randn(NumcCtx *ctx, const size_t *shape,
                            size_t dim, NumcDType dtype);

Creates an array with standard normal N(0, 1) values using the Box-Muller transform:

• Float types: true N(0, 1) samples.
• Integer types: N(0, 1) doubles rounded and cast to the integer type (most values will be 0 or +/-1).

// 4x4 standard normal samples
NumcArray *n = numc_array_randn(ctx, (size_t[]){4, 4}, 2, NUMC_DTYPE_FLOAT64);

Neural Network Weight Initialization

These functions implement common weight initialization strategies used in deep learning.

He (Kaiming) Initialization

NumcArray *numc_array_random_he(NumcCtx *ctx, const size_t *shape,
                                size_t dim, NumcDType dtype, size_t fan_in);

Draws from N(0, sqrt(2 / fan_in)). Recommended for layers followed by ReLU activations.

// Initialize weights for a layer with 784 inputs
size_t shape[] = {784, 256};  // 784 inputs, 256 outputs
NumcArray *w = numc_array_random_he(ctx, shape, 2,
                                     NUMC_DTYPE_FLOAT32,
                                     /*fan_in=*/784);

fan_in is the number of input units. For a fully-connected layer with in inputs, fan_in = in. For a convolutional layer, fan_in = in_channels * kernel_height * kernel_width.

Xavier (Glorot) Initialization

NumcArray *numc_array_random_xavier(NumcCtx *ctx, const size_t *shape,
                                    size_t dim, NumcDType dtype,
                                    size_t fan_in, size_t fan_out);

Draws from a uniform distribution [-limit, limit) where limit = sqrt(6 / (fan_in + fan_out)). Recommended for layers with tanh or sigmoid activations.

// Initialize weights for a 256 -> 128 layer
size_t shape[] = {256, 128};
NumcArray *w = numc_array_random_xavier(ctx, shape, 2,
                                         NUMC_DTYPE_FLOAT32,
                                         /*fan_in=*/256,
                                         /*fan_out=*/128);

Example: Simple Neural Network Layer

NumcCtx *ctx = numc_ctx_create();

size_t batch = 32, in = 784, out = 256;

// Initialize weights and biases
NumcArray *W = numc_array_random_he(ctx, (size_t[]){in, out}, 2,
                                     NUMC_DTYPE_FLOAT32, in);
NumcArray *b = numc_array_zeros(ctx, (size_t[]){1, out}, 2,
                                 NUMC_DTYPE_FLOAT32);

// Input batch
NumcArray *X = numc_array_rand(ctx, (size_t[]){batch, in}, 2,
                                NUMC_DTYPE_FLOAT32);

// Forward pass: Y = X @ W + b (with broadcasting on b)
NumcArray *Y = numc_array_create(ctx, (size_t[]){batch, out}, 2,
                                  NUMC_DTYPE_FLOAT32);
numc_matmul(X, W, Y);
numc_add(Y, b, Y);  // b broadcasts from (1, 256) to (32, 256)

// ReLU: Y = max(Y, 0)
numc_clip_inplace(Y, 0.0, 1e30);

numc_ctx_free(ctx);