hike-lang

Cgo-free Native FFI: Hike to Go Direct Linking Architecture

This document provides technical specifications and operational workflows for writing native C-compatible functions (cfunc) in Hike, compiling them to LLVM IR and object files (.syso), and statically linking them directly into Go binaries without Cgo for ultra-low-latency execution.


1. Architecture Overview

Interfacing Go with C traditionally relies on Cgo. However, Cgo introduces heavy C toolchain dependencies, complicates cross-compilation, and incurs measurable invocation overhead due to stack switching (entersyscall / exitsyscall) between Goroutine stacks and OS system stacks.

This architecture achieves 100% Cgo-free native execution (CGO_ENABLED=0) by combining three core mechanisms:

[ Hike Source (.go.hike) ]
         │
         ▼ (hikec go)
[ LLVM IR (.ll) ]
         │
         ▼ clang -O2 -mno-stack-arg-probe
[ .syso Object ] ──────┐
                       │  (Go Linker Static Merge)
[ Plan 9 Stub (.s) ] ──┼─────────────────────────► [ Go Binary (main.exe) ]
                       │
[ Go Wrapper (.go) ] ──┘


2. Core Technologies and Mechanisms

2.1 Go Linker’s .syso Automatic Merging

The Go toolchain (cmd/link) features a built-in mechanism: when an object file named <name>_<GOOS>_<GOARCH>.syso (e.g., fizzlib_windows_amd64.syso) is present in a package directory, **the linker statically merges the COFF/ELF native machine code directly into the target binary even with CGO_ENABLED=0**.

2.2 Plan 9 Assembly ABI Bridge

Go’s internal calling convention (Go ABI) is incompatible with standard C ABIs (such as the Windows x64 ABI or System V AMD64 ABI). The Hike compiler generates a Plan 9 assembly bridge (stub_windows_amd64.s) to reorder argument registers, set up stack shadow space, and jump to the native symbol (c_<Name>).

2.3 LLVM / Clang Pure Machine Code Sanitization (Zero libc Dependency)

Go’s pure linker cannot resolve symbols against the C runtime library (libc) or CRT. A single unresolved external reference results in link failure.

2.4 Stack Models and passthrough cfunc

Goroutines operate on dynamic, minimal initial stacks (2 KB to 4 KB).


3. Project Structure

myproject/
├── fizzlib/
│   ├── fizzlib.go.hike            # Hike native implementation
│   ├── fizzlib.go                 # Idiomatic Go public API & wrapper
│   ├── stub_windows_amd64.s       # [Auto-generated] Plan 9 assembly stub
│   ├── fizzlib_windows_amd64.syso # [Auto-generated] Native machine code object
│   └── fizzlib.ll                 # [Intermediate] LLVM IR
├── main.go                        # Application entry point
└── Makefile                       # Build orchestration


4. Implementation Code

4.1 Hike Source: fizzlib/fizzlib.go.hike

Use the cfunc keyword to define exported, C-ABI-compliant native function symbols.

package fizzlib

// Standard cfunc
cfunc GetFizzFileSize(fn_ptr cstring, fn_len int) int {
    return 15
}

cfunc ReadFizzFile(fn_ptr cstring, fn_len int, readSize int, buf *byte) int {
    if readSize <= 0 {
        return 0
    }
    for i := 0; i < readSize; i++ {
        buf[i] = byte(65 + (i % 26))
    }
    return readSize
}

// passthrough: Executes directly on Goroutine stack with near-zero latency
passthrough cfunc GetMetaData(fn_ptr cstring, fn_len int, outLen *int) cstring {
    *outLen = 4
    return "fizz"
}

cfunc free(ptr *byte) {
}

4.2 Generated Plan 9 Stub: fizzlib/stub_windows_amd64.s

Generated automatically by hikec go.

// Code generated by hikec go. DO NOT EDIT.
#include "textflag.h"

// GetFizzFileSize (passthrough: false)
TEXT ·_hike_GetFizzFileSize(SB), 0, $32-24
    MOVQ ptr_arg+0(FP), CX
    MOVQ len_arg+8(FP), DX
    SUBQ $32, SP
    CALL c_GetFizzFileSize(SB)
    ADDQ $32, SP
    MOVQ AX, ret+16(FP)
    RET

// GetMetaData (passthrough: true)
TEXT ·_hike_GetMetaData(SB), NOSPLIT, $32-32
    MOVQ fn_ptr_arg+0(FP), CX
    MOVQ fn_len_arg+8(FP), DX
    MOVQ outLen_arg+16(FP), R8
    SUBQ $32, SP
    CALL c_GetMetaData(SB)
    ADDQ $32, SP
    MOVQ AX, ret+24(FP)
    RET

4.3 Go Wrapper: fizzlib/fizzlib.go

Exposes idiomatic Go types (string, slices) while managing pointers passed into internal assembly stubs.

package fizzlib

import "unsafe"

// Internal Plan 9 stub declarations
//go:noescape
func _hike_GetFizzFileSize(ptr unsafe.Pointer, length int) int

//go:noescape
func _hike_ReadFizzFile(fn unsafe.Pointer, fnLen int, readSize int, buf unsafe.Pointer) int

//go:noescape
func _hike_GetMetaData(fn unsafe.Pointer, fnLen int, outLen *int) unsafe.Pointer

//go:noescape
func _hike_free(ptr unsafe.Pointer)

// Public Go API
func GetFizzFileSize(filename string) int {
    p := unsafe.StringData(filename)
    return _hike_GetFizzFileSize(unsafe.Pointer(p), len(filename))
}

func ReadFizzFile(filename string, readSize int, buf []byte) int {
    p := unsafe.StringData(filename)
    var bp unsafe.Pointer
    if len(buf) > 0 {
        bp = unsafe.Pointer(&buf[0])
    }
    return _hike_ReadFizzFile(unsafe.Pointer(p), len(filename), readSize, bp)
}

func GetMetaData(filename string) string {
    p := unsafe.StringData(filename)
    var outLen int
    resPtr := _hike_GetMetaData(unsafe.Pointer(p), len(filename), &outLen)
    if resPtr == nil || outLen == 0 {
        return ""
    }
    return string(unsafe.Slice((*byte)(resPtr), outLen))
}

4.4 Consumer: main.go

package main

import (
    "fmt"
    "examples_gohike/fizzlib"
)

func main() {
    size := fizzlib.GetFizzFileSize("test.txt")
    fmt.Printf("[1] Fetched size: %d bytes\n", size)

    buf := make([]byte, size)
    n := fizzlib.ReadFizzFile("test.txt", size, buf)
    fmt.Printf("[2] Read result (len=%d): %s\n", n, string(buf))

    meta := fizzlib.GetMetaData("test.txt")
    fmt.Printf("[3] Metadata: %s\n", meta)
}


5. Build and Execution Workflow

Step 1: Generate .syso and Assembly Stubs with Hike

Run hikec go -v pointing to the target package directory:

hikec.exe go -v ./fizzlib

Build Output:

[hikec go] Found 1 .go.hike file(s) in ...\fizzlib
  - fizzlib.go.hike
[LOADER] Parsing file: ...\fizzlib\fizzlib.go.hike
[PARSER] [1:9] Declared package 'fizzlib'
[PARSER] [4:1] Parsing cfunc: GetFizzFileSize
[PARSER] [9:1] Parsing cfunc: ReadFizzFile
[PARSER] [22:13] Parsing cfunc: GetMetaData
[PARSER] [28:1] Parsing cfunc: free
[hikec go] Generating Plan 9 assembly stubs for 4 cfunc(s):
  - GetFizzFileSize (normal)
  - ReadFizzFile (normal)
  - GetMetaData (passthrough (NOSPLIT))
  - free (normal)
[hikec go] Successfully generated: ...\fizzlib\stub_windows_amd64.s
[hikec go] Running: clang -c -O2 -mno-stack-arg-probe ...\fizzlib\fizzlib.ll -o ...\fizzlib\fizzlib_windows_amd64.syso --target=x86_64-w64-windows-gnu
[hikec go] Successfully generated: ...\fizzlib\fizzlib_windows_amd64.syso

Step 2: Direct Execution in Go

Run or build the Go application with pure static linking:

go run main.go

Execution Result:

[1] Fetched size: 15 bytes
[2] Read result (len=15): ABCDEFGHIJKLMNO
[3] Metadata: fizz


6. Constraints and Operational Rules

  1. Strict Prohibition of Standard libc Calls Functions compiled into .syso cannot reference standard C library symbols (e.g., printf, malloc). Memory buffers must be allocated in Go and passed down as slices or pointers.
  2. **Stack Budget for passthrough cfunc** Functions marked with passthrough execute directly on Goroutine stacks (~2 KB). Allocating large local arrays (such as [4096]byte) or executing unbounded recursion will trigger silent stack corruption or immediate segmentation faults. Keep stack frames minimal.