User:TomYu/Plugin support improvements - Revision history

TomYu at 18:58, 3 August 2010

2010-08-03T18:58:35Z

TomYu at 20:23, 20 July 2010

2010-07-20T20:23:01Z

Show changes

TomYu: add interactions, config

2010-07-08T23:01:52Z

add interactions, config

TomYu at 21:40, 7 July 2010

2010-07-07T21:40:23Z

Show changes

TomYu: example PLM header

2010-07-06T16:07:54Z

example PLM header

TomYu: partial rewrite checkpoint

2010-07-06T03:17:24Z

partial rewrite checkpoint

Show changes

TomYu: copy project page

2010-07-01T19:58:44Z

copy project page

New page

{{project-early}}

This page contains some notes about improving the plugin infrastructure, but does not make a specific proposal yet.

Some possibly useful ideas at [http://www.drdobbs.com/cpp/204202899 Dr Dobb's]. It's rather biased toward C++, and we may disagree with some of the details.

==Motivation ==

Have you tried to write a new plugin for Kerberos? What was your experience? Your answer to this question is the motivation for this project.
We set up our goal on creating a framework that

* Facilitates simple and clear additions of new plugin interfaces and new implementations of the existing plugins;
* Handles both built-in and dynamic plugins;
* Allows multiple implementation of the same plugin interface;
* Provides uniform way to supply parameters for plugin configuration;

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (Dependent library shared objects of the calling library can contain "built-in" modules for the calling library.) The distinguishing feature of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

==Structure==
[[Image:Slide_v2.jpg]]

==Participants==

* Plugin Manager ('''PM''') - Abstract module that declares interface for operations that manage plugin configuration and registry services;
* Plugin Manager Implementation ('''PMI''') - Implements the operations for plugin configuration and registry services;
* Plugin Loader ('''PL''') - Abstract module that declares an interface for operations that create abstract plugin objects. It knows about:
:: Set of available implementations;
:: How to create them;
:: We might want to have a separate loader function for each pluggable interface, for type safety;
* Plugin Loader Implementation ('''PLI''') - Concrete implementation of PL;
* Plugin Interface ('''PI''') - Abstract module that declares plugin interface.
* Plugin Interface Implementation ('''PII''') - Concrete implementation of the plugin interface;
* Caller - Plugin caller.

==Collaboration==

* An instance ''pl_manager'' of a particular PMI is created as part of ''krb5_init_context'' operation at run-time (1). This instance will use methods specific to PMI to configure and register the desired plugin implementations;
* The list of the desired plugin implementations is known to the specific PLI which is aggregated with PMI (2);
* PMI defers acquiring of plugin interfaces to PLI (3);
* The client uses ''pl_manager'' to invoke the desired plugin interface (4).

==User Interface==

; plhandle plugin_manager_get_service (plugin_manager* instance,const char*, const int);
; int plugin_manager_configure (plugin_manager* instance,const char*);
; void plugin_manager_start (plugin_manager* instance);
; void plugin_manager_stop (plugin_manager* instance);

; void get_loader_content (loader_handle handle, const char* container[]);
; plhandle create_api (loader_handle handle, const char* plugin_name);

==Implementation==

; Proof of concept code is available from ''plugins'' branch: ''svn+ssh://svn.mit.edu/krb5/branches/plugins''

; k5-int.h: ''plugin_manager *pl_manager'' is added to _krb5_context

; PMI vs PLI: In general a particular PMI is tied to one or more PLIs. In its turn, each PLI is responsible for loading one or more plugins. The notion of the multiple plugins per PLI is justified by the convenience of the plugin grouping based on their relativity.

Implementation must be MT safe

==Consequences==

The management details, such as configuration and loading, are hidden from the plugin writers and callers.

* The responsibility of the writer of the new plugin interface (PI) is limited to the writing source and header as described at Template_1:

''Template 1''

#include <plugin_manager.h>
#include <k5-int.h>
typedef struct {
int version;
krb5_error_code (*fn1)(krb5_context);
} plugin_PLName;

krb5_error_code plugin_PLName_fn1(plhandle handle, krb5_context)
{
plugin_PLName* api = (plugin_PLName*) handle.api;
api->plugin_PLName_fn1(context);
return 0;
}

* The responsibility of the writer of the new plugin interface implementation (PII) is limited to the writing source and header as described at Template_2 and the notifying the plugin loader about its availability Template_3:

''Template 2''

static krb5_error_code
plugin_PLName_fn1(krb5_context ctx, krb5_data *data)
{ /*intended functionality */}

static void
plugin_PLName_cleanup(plugin_PLName* api)
{ /* do some cleanup */ }

static krb5_error_code
plugin_PLName_init(void)
{ /* do some init */ }

plhandle plugin_PLName_PLImpl_create()
{
plhandle handle;
plugin_PLName* api = malloc(sizeof(plugin_prng));
api->version = 0;
api->fn1 = plugin_PLName_fn1;
api->PLName_init = plugin_PLName_init;
api->PLName_cleanup = plugin_PLName_cleanup;
handle.api = api;
return handle;
}

''Template 3''

static plugin_descr _table[] = {
{"plugin_PLImpl_PLName", plugin_PLName_PLImpl_create},
};

''For better readability some of the verification, clean-up, error handling code is omitted from the templates''

==Sample Code==

===Plugin initialization===
The following is an example of the plugin initialization:

plugin_default_manager_get_instance(&plugin_mngr_instance);
plugin_manager_configure(ctx->pl_manager, conf_path);
plugin_manager_start(ctx->pl_manager);

===How to call plugins===
The following is an example of invoking Password Quality plugin:

plhandle plugin_handle;
plugin_handle = plugin_manager_get_service(ctx->pl_manager, "plugin_pwd_qlty", PWD_QLTY_KRB);
plugin_pwd_qlty_check(ctx->plugin_manager, srv_handle, password, use_policy, pol, principal);

===How to construct plugin interface (PI)===
The following is an example of plugin interface called "password quality":

typedef struct {
int version;
kadm5_ret_t (*pwd_qlty_init)(kadm5_server_handle_t);
void (*pwd_qlty_cleanup)();
kadm5_ret_t (*pwd_qlty_check)(kadm5_server_handle_t, char*, int, kadm5_policy_ent_t, krb5_principal);
} plugin_pwd_qlty;

kadm5_ret_t
plugin_pwd_qlty_check( plhandle handle, kadm5_server_handle_t srv_handle, char *password, int use_policy, kadm5_policy_ent_t pol, krb5_principal principal)
{
plugin_pwd_qlty* api = (plugin_pwd_qlty*) handle.api;
api->pwd_qlty_check(srv_handle, password, use_policy, pol, principal);
}

kadm5_ret_t
plugin_pwd_qlty_init(plhandle, kadm5_server_handle_t)
{ /* do some init */ }

void
plugin_pwd_qlty_cleanup(plhandle)
{ /* do some clean-up */ }

===How to construct plugin interface implementation (PII)===
Every PII should have only one public ''create'' function. For PII called "password quality krb" it would be ''plhandle plugin_pwd_qlty_krb_create()''.The following is an example of the this PII:

plhandle plugin_pwd_qlty_krb_create()
{
plhandle handle;
plugin_pwd_qlty* api = malloc(sizeof(plugin_pwd_qlty));
api->version = 1;
api->pwd_qlty_init = plugin_pwd_qlty_init;
api->pwd_qlty_check = plugin_pwd_qlty_check;
api->pwd_qlty_cleanup = plugin_pwd_qlty_clean;
handle.api = api;
return handle;
}

static kadm5_ret_t
plugin_pwd_qlty_check(kadm5_server_handle_t srv_handle, char *password, int use_policy, kadm5_policy_ent_t pol, krb5_principal principal)
{ /* do some quality verification */ }

static kadm5_ret_t
plugin_pwd_qlty_init(kadm5_server_handle_t handle)
{ /* do some initialization */ }

static kadm5_ret_t
plugin_pwd_qlty_cleanup()
{ /* do some cleanup */ }

''For better readability some of the verification, clean-up, error handling code is omitted from the samples''

==Built-in Plugins==

*; Examples: PreAuth, AuthData, Password Quality (see ''Current plugins'' section for details)

*; Invoking multiple implementations of the same plugin: This is achieved by using the aimed ''plugin_id'''s in ''plugin_manager_get_service'' calls. For example, two specific implementations of the password quality plugins may be invoked in password check. Alternatively, one can use '''all available''' plugin implementations as it is done for preauthentication.

==Dynamic Plugins==

*; Examples: Audit, Password sync

*; Requirements:
# The difference between dynamic and built-in plugins should be on the level of linkage. Their codebase should be identical.
# Plugin consumer should not have the knowledge about the type of plugins it is using (in terms of dynamic/built-in). On the caller side the code invoking plugin should stay the same for both dynamic and built-in plugins.
# Similar to built-in plugins, the plugin consumer may call the multiple implementations of the same plugin interface. For example, two implementations of the audit plugin may be used to handle password change and intruder lockout events.
# It is responsibility of the administrator to point to the location of the dynamic plugin as part of the configuration.

*; Creating dynamic plugin: The appropriate loader is linked with the set of the desired plugins resulting into the shared library. Details:
# One of the deliverables of the plugin framework is the loader file. Its responsibility is to load modules that are listed in its local table;
# The dynamic plugin creator must fill this table with the list of the desired plugins.
# The dynamic plugin creator then links the loader with the set of the desired plugins resulting into the shared library;
# Administrator points to the location of the dynamic plugin by setting ''plugin_loader_path'' configuration attribute to this library.

''The following is the example of the table in the loader:''
static plugin_descr plugin_dyn_factory_table[] = {
{"plugin_pwd_qlty_DYN", plugin_pwd_qlty_DYN_create},
{"plugin_pwd_qlty_DYN_X", plugin_pwd_qlty_DYN_X_create},
{NULL,NULL}
};

*; Invoking dynamic plugin: Similar to builtin plugins with the following additions. The option ''plugin_loader_path'' in the configuration file refers to the location of the dynamic plugin shared library and the option ''plugin_loader_type'' must be set to ''dynamic''.

==Configuration==

===Configuration file attributes===
:: ''plugin list'' - list of the desired plugins;
:: ''plugin_api'' - name of the plugin interface;
:: ''plugin_name'' - concrete plugin implementation;
:: ''plugin_id'' - the numeric id that plugin implementation is associated with. Used by the caller;
:: ''plugin_loader_name'' - name of the loader the particular implementation is associated with;
:: ''plugin_loader_type'' - defines the plugin type: built-in or dynamic;
:: ''plugin_loader_path'' - path to the plugin dynamic library

===Example of the plugin section in krb5.conf===
[plugins]
/* built-in plugin */
plugin_list = PQ1
PQ1 = {
plugin_api = plugin_pwd_qlty
plugin_loader_name = plugin_default_factory
plugin_loader_type = static
plugin_name = plugin_pwd_qlty_krb
plugin_id = 1
}

/* dynamic plugin */
plugin_list = PQ_DYN
PQ_DYN = {
plugin_api = plugin_pwd_qlty
plugin_floader_name = plugin_dyn_factory
plugin_loader_type = dynamic
plugin_name = plugin_pwd_qlty_DYN
plugin_loader_path = /path-to-the-lib/libplugin_dynamic.so
plugin_id = 33
}

===Alternatives===
The choice of plugin configuration format is a matter of taste. It may be presented in XML, YAML, JSON etc. One needs only an appropriate parser.

===Configuration change at run-time===
(...)

==Directory Structure==

*;src/plugin_core: plugin_manager.[ch], plugin_loader.[ch], impl/plugin_XXX_manager.[ch], impl/plugin_XXX_loader.[ch]
*;src/plugins: pluginA/plugin_A.[ch], pluginA/plugin_A_implN/plugin_A_implN.[ch]
*;src/plugin_dynamic: plugin_dyn_factory.[ch]

==Defaults==

(...)

==Build process==

*; Options: DEBUG_PLUGINS - output more info for debugging

==Current plugins==

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

==Future plugins==

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

==Current support infrastructure==

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

==Problem areas==

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

← Older revision		Revision as of 18:58, 3 August 2010
Line 1:		Line 1:
−	{{project-target \| 1.9}}
−
−	{{project-early}}
−
	The MIT Kerberos implementation has a number of places where modular extensibility would facilitate development and deployment of new technologies. Developers of the core code and authors of extension code have an interest in making it easier for third parties to extend the code in a well-compartmentalized way. Administrators of Kerberos deployments and packagers of software have an interest in making it easy to deploy new capabilities in a simple way.		The MIT Kerberos implementation has a number of places where modular extensibility would facilitate development and deployment of new technologies. Developers of the core code and authors of extension code have an interest in making it easier for third parties to extend the code in a well-compartmentalized way. Administrators of Kerberos deployments and packagers of software have an interest in making it easy to deploy new capabilities in a simple way.

@@ Line 72: / Line 72: @@
 Although the caller (actually the plugin support code) allocates the vtable structure in the above description, one alternative is to have the module perform the allocation of the structure itself.  This can cause problems if the module uses a different memory allocator than the caller.
-Should the single function symbol have a prefix that depends on the  name of the plugin module?
+Should the single function symbol have a prefix that depends on the name of the plugin module?
 ;Yes: allows identical module source code to be used for the built-in and loadable module versions of a module
@@ Line 79: / Line 79: @@
 ===== Built-in module provider interface =====
-A built-in module provides the same interface as a loadable module.  In the alternative where we use same exported function symbol for each loadable module implementing a pluggable interface, the built-in modules will still need distinct prefixes are required for each vtable retrieval function.  This requires slightly different source code for a built-in module versus a loadable module.
+A built-in module provides the same interface as a loadable module. In the alternative where we use same exported function symbol for each loadable module implementing a pluggable interface, the built-in modules will still need distinct prefixes are required for each vtable retrieval function.  This requires slightly different source code for a built-in module versus a loadable module.
 ===== Alternative: one symbol per method =====
@@ Line 91: / Line 91: @@
 === Plugin manager ===
-The plugin manager (PLM) provides a set of generic support capabilities that are independent of individual pluggable interfaces.  It centralizes the discovery process for plugin modules.  Typically, consumers of pluggable interfaces do not call it directly, but instead call a loader function of the pluggable interface, which in turn calls the plugin manager.  The plugin manager works in concert with a set of plugin registries to locate plugin modules.
+The plugin manager (PLM) provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules.  Typically, consumers of pluggable interfaces do not call it directly, but instead call a loader function of the pluggable interface, which in turn calls the plugin manager.  The plugin manager works in concert with a set of plugin registries to locate plugin modules.
 The <code>krb5_init_context()</code> functions will create a plugin manager context that will exist in the krb5_context, and add the initial plugin registries to the plugin manager context.  An application may use <code>krb5_get_plm()</code> to retrieve the plugin manager context so it can register its own built-in plugin modules.
@@ Line 109: / Line 109: @@
 Initially, the plugin framework implementation will contain two types of registry:
-; built-in module table registry: locates built-in plugin modules.  This also allows an application to register its private built-in plugin modules.
+; built-in module table registry: locates built-in plugin modules. This also allows an application to register its private built-in plugin modules.
 ; loadable module directories registry: locates dynamically loadable plugin modules.  It maps plugin interface identifiers to directories where modules implementing that interface can be found, and can thereby locate individual modules.
@@ Line 126: / Line 126: @@
 '''NEEDS MORE DETAIL'''
 == Details ==

@@ Line 57: / Line 57: @@
 === Plugin manager ===
-The plugin manager (PLM) provides a set of generic support capabilities that are independent of individual pluggable interfaces.  It centralizes the discovery process for plugin modules.
+The plugin manager (PLM) provides a set of generic support capabilities that are independent of individual pluggable interfaces.  It centralizes the discovery process for plugin modules.  Typically, consumers of pluggable interfaces do not call it directly, but instead call a loader function of the pluggable interface.
 * given the identifier of a pluggable interface, return an opaque handle representing the entire collection of available plugin modules that implements that pluggable interface.
@@ Line 69: / Line 69: @@
 === Caller (consumer) interface ===
-A caller (consumer) of a pluggable interface uses an opaque handle to call the methods of a plugin module.  Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name.  One rationale for doing this instead of having the caller invoke methods through a function pointer is that it makes it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)
+A caller (consumer) of a pluggable interface uses an opaque handle to call the methods of a plugin module.  Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name.  Conceptually, these pluggable interface functions are wrapper functions that call through function pointers contained in the opaque handle object. (though this is not the only possible implementation)
 A handle can represent:
@@ Line 97: / Line 97: @@
 Should the single function symbol have a prefix that depends on the  name of the plugin module?
-;Yes: could allow identical code to be used for the built-in and loadable module versions of a module
+;Yes: allows identical module source code to be used for the built-in and loadable module versions of a module
 ;No: adding a prefix to the symbol adds complexity to the code that looks up the symbol
 ==== Built-in module provider interface ====
-A built-in module provides the same interface as a loadable module, except distinct prefixes are required for each vtable retrieval function.
+A built-in module provides the same interface as a loadable module.  In the alternative where we use same exported function symbol for each loadable module implementing a pluggable interface, the built-in modules will still need distinct prefixes are required for each vtable retrieval function.
 ==== One-symbol-per-method alternative loadable module provider interface ====
@@ Line 186: / Line 186: @@
 == Links ==
-Some possibly useful ideas at [http://www.drdobbs.com/cpp/204202899 Dr Dobb's]. It's rather biased toward C++, and we may disagree with some of the details.
+Some possibly useful ideas at [http://www.drdobbs.com/cpp/204202899 Dr Dobb's]. It's rather biased toward C++, and we may disagree with some of the details.  Among other things, it advocates separating the domain-specific interface of a plugin module from its domain-independent interface: plugin modules get passed a handle for the "registry services" of the plugin manager, and the modules must call those services to register their objects or methods.