How to Call Python Callbacks from Plain Rust Functions #6105
Description
1.Register a Python callback in Rust via #[pyfunction]
2.Later invoke that callback from a plain Rust function (not a #[pyfunction])
3.Do this safely across FFI boundaries (e.g., when called from C)
Challenges
1.VirtualMachine Lifetime: The VirtualMachineinstance can't be stored globally
2.Thread Safety: Python callbacks and VM state have thread-safety requirements
3.FFI Compatibility: Need to expose simple C-compatible functions
Solution Architecture
- Thread-Safe Callback Storage
use std::sync::{Arc, Mutex};
use rustpython_vm::{VirtualMachine, PyObjectRef};
type PyCallback = Arc<Mutex<Option<(PyObjectRef, Arc)>>>;
static GLOBAL_CALLBACK: Lazy = Lazy::new(|| Arc::new(Mutex::new(None)));
2. Registration Function (#[pyfunction])
#[pyfunction]
fn register_callback(callback: PyObjectRef, vm: &VirtualMachine) -> PyResult<()> {
// Store both callback and VM reference
let mut guard = GLOBAL_CALLBACK.lock().unwrap();
*guard = Some((
callback.clone(),
Arc::new(vm.shallow_clone()) // Create a thread-safe clone
));
Ok(())
}- Plain Rust Invocation Function
#[no_mangle]
pub extern "C" fn invoke_callback(message: *const c_char) -> *mut c_char {
let msg = unsafe { CStr::from_ptr(message) }.to_str().unwrap();
let guard = GLOBAL_CALLBACK.lock().unwrap();
if let Some((callback, vm)) = &*guard {
let args = (msg.to_string(),);
match callback.call(args, &vm) {
Ok(result) => {
let result_str = result.to_string();
CString::new(result_str).unwrap().into_raw()
}
Err(e) => {
CString::new(format!("Error: {}", e)).unwrap().into_raw()
}
}
} else {
CString::new("No callback registered").unwrap().into_raw()
}
}- Python Usage Example
import rust_py_module
from ctypes import c_char_p, CDLL
lib = CDLL("./your_rust_library.so")
def my_callback(message):
print(f"Python received: {message}")
return "Success from Python"
# Register the callback
rust_py_module.register_callback(my_callback)Simulate C calling Rust
msg = b"Hello from C!"
result = lib.invoke_callback(msg)
print(f"Callback result: {result}")How can I achieve the above function or does rustpython not support this yet?