Metadata-Version: 2.1
Name: refcount
Version: 0.9
Summary: A Python package for reference counting and interop with native pointers
Home-page: https://github.com/csiro-hydroinformatics/pyrefcount
Author: Jean-Michel Perraud
Author-email: per202@csiro.au
License: UNKNOWN
Project-URL: Bug Reports, https://github.com/csiro-hydroinformatics/pyrefcount/issues
Project-URL: Source, https://github.com/csiro-hydroinformatics/pyrefcount
Description: # refcount
        
        [![license](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/csiro-hydroinformatics/pyrefcount/blob/master/LICENSE.txt) ![status](https://img.shields.io/badge/status-beta-blue.svg) [![Documentation Status](https://readthedocs.org/projects/pyrefcount/badge/?version=latest)](https://pyrefcount.readthedocs.io/en/latest/?badge=latest) master: [![Build status - master](https://ci.appveyor.com/api/projects/status/vmwq7xarxxj8s564/branch/master?svg=true)](https://ci.appveyor.com/project/jmp75/pyrefcount/branch/master) testing: [![Build status - devel](https://ci.appveyor.com/api/projects/status/vmwq7xarxxj8s564/branch/testing?svg=true)](https://ci.appveyor.com/project/jmp75/pyrefcount/branch/testing)
        
        
        ![Reference counted native handles](./docs/img/refcount-principles.png "Reference counted native handles")
        
        This package is primarily for managing resources in native libraries, written for instance in C++, from Python. While it boils down to "simply" maintaining a set of counters, **it is deceptively complicated to do so properly** and not end up with memory leaks or crashes. This package offers structured options for reliably managing external native resources. Surprisingly I could not locate an existing package doing just what I needed. Other use cases requiring reference counting, aside from native library resources, may benefit from reusing and extending classes in `refcount`.
        
        `refcount` from version 0.7 includes classes using [cffi](https://cffi.readthedocs.io/). Other low-level interoperability mechanisms may well be added in the future.
        
        ## License
        
        MIT (see [License.txt](https://github.com/csiro-hydroinformatics/pyrefcount/blob/master/LICENSE.txt))
        
        ## Documentation
        
        Hosted at [pyrefcount.readthedocs.io](https://pyrefcount.readthedocs.io/en/latest/?badge=latest)
        
        ## Source code
        
        The code repository is on [GitHub](https://github.com/csiro-hydroinformatics/pyrefcount).
        
        ## Installation
        
        ```sh
        pip install refcount
        ```
        
        From source:
        
        ```sh
        pip install -r requirements.txt
        python setup.py install
        ```
        
        ## Sample use
        
        The following example is based on one of the unit tests. 
        
        Say we have a C++ library with objects and a C API:
        
        ```C++
        #define TEST_DOG_PTR  testnative::dog*
        #define TEST_OWNER_PTR  testnative::owner*
        #define TEST_COUNTED_PTR  testnative::reference_counter*
        
        testnative::dog* create_dog();
        testnative::owner* create_owner(testnative::dog* d);
        void say_walk(testnative::owner* owner);
        void release(testnative::reference_counter* obj);
        // etc.
        ```
        
        From the outside of the library the API is exported with opaque pointers `void*` (C structs pointers and native C99 types could be handled too).
        
        ```C++
        void* create_dog();
        void* create_owner(void* d);
        void say_walk(void* owner);
        void release(void* obj);
        // etc.
        ```
        
        Starting with the end in mind, from Python we want an API hiding the low level details close to the C API, in particular avoiding managing native memory via `release` C API calls, piggybacking the python GC instead.
        
        ```py
        dog = Dog()
        owner = DogOwner(dog)
        owner.say_walk()
        print(dog.position)
        dog = None # the "native dog" is still alive though, as the owner incremented the ref count
        owner = None
        ```
        
        This is doable with `refcount` and the `cffi` package. One possible design is:
        
        ```py
        ut_ffi = cffi.FFI()
        
        ut_ffi.cdef('extern void* create_dog();')
        ut_ffi.cdef('extern void* create_owner( void* d);')
        ut_ffi.cdef('extern void say_walk( void* owner);')
        ut_ffi.cdef('extern void release( void* obj);')
        # etc.
        
        ut_dll = ut_ffi.dlopen('c:/path/to/test_native_library.dll', 1) # Lazy loading
        
        class CustomCffiNativeHandle(CffiNativeHandle):
            def __init__(self, pointer, prior_ref_count = 0):
                super(CustomCffiNativeHandle, self).__init__(pointer, type_id='', prior_ref_count = prior_ref_count)
        
            def _release_handle(self) -> bool:
                ut_dll.release(self.get_handle())
                return True
        
        class Dog(CustomCffiNativeHandle):
            def __init__(self, pointer = None):
                if pointer is None:
                    pointer = ut_dll.create_dog()
                super(Dog, self).__init__(pointer)
            # etc.
        
        class DogOwner(CustomCffiNativeHandle):
        
            def __init__(self, dog):
                super(DogOwner, self).__init__(None)
                self._set_handle(ut_dll.create_owner(dog.get_handle()))
                self.dog = dog
                self.dog.add_ref() # Do note this important reference increment
        
            def say_walk(self):
                ut_dll.say_walk(self.get_handle())
        
            def _release_handle(self) -> bool:
                super(DogOwner, self)._release_handle()
                # super(DogOwner, self)._release_handle()
                self.dog.release()
                return True
        ```
        
        ## Related work
        
        ### Ancestry, acknowledgements
        
        This python package `refcount` actually spawned from prior work for interoperability between C++, R and .NET ([R.NET](https://github.com/rdotnet/rdotnet))
        
        `refcount` features using `cffi` were also significantly informed by Kevin Plastow's [work](https://search.informit.com.au/documentSummary;dn=823898220073899;res=IELENG) while he was at the Australian Bureau of Meteorology; this contribution is gratefully acknowledged.
        
        In you have native interop needs you may also want to look at:
        
        * the nuget package [dynamic-interop-dll](https://github.com/rdotnet/dynamic-interop-dll) for .NET/native interop.
        * a set of mostly c++ software [tools for interop with C/C++](https://github.com/csiro-hydroinformatics/rcpp-interop-commons)
        * a C# library for [generating interop glue code on top of C API glue code](https://github.com/csiro-hydroinformatics/c-api-wrapper-generation).
        
        ### Other python packages
        
        `refcount` was created in part because no existing prior (Python) work could quite fit the need. There are however packages that may better address your particular need:
        
        * [infi.pyutils](https://pypi.org/project/infi.pyutils/) contains a reference counting class.
        
Keywords: interop python native-libraries reference-counting handle cffi
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development 
Classifier: Topic :: Software Development :: Object Brokering
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Description-Content-Type: text/markdown
