copyninja · August 29, 2015 13:58 · Apr 7, 2014 · Apr 7, 2014 · Apr 7, 2014 · Apr 7, 2014
diff --git a/dyanmic_module_loading.rst b/dyanmic_module_loading.rst
@@ -52,12 +52,14 @@ python files lying around in a directory, which I wanted to plug into
 the bot during run time and use them depending on some conditions. So
 basic structure which I was looking was as follows.
 
-   pluginloader.py
-   plugins
-     |
-     |__ aplugin.py
-     |
-     |__ bplugin.py
+::
+
+pluginloader.py
+plugins
+  |
+  |__ aplugin.py
+  |
+  |__ bplugin.py
 
 `pluginloader.py` is the file which needs to load python files under
 `plugins` directory. This was again done using `importlib` module as

diff --git a/dyanmic_module_loading.rst b/dyanmic_module_loading.rst
@@ -18,8 +18,7 @@ Case 1: Loading installed python module
 ---------------------------------------
 
 In case of SILPA I need to load pre-installed modules and here is the
-`old code
-<https://github.com/Project-SILPA/Silpa-Flask/blob/master/core/modulehelper.py#L24>_`
+`old code <https://github.com/Project-SILPA/Silpa-Flask/blob/master/core/modulehelper.py#L24>_`
 , that is a bit hacky code I copied from Active State Python
 recipies. I found a bit better way to do it using `importlib` module
 as shown below.

diff --git a/dyanmic_module_loading.rst b/dyanmic_module_loading.rst
@@ -25,6 +25,7 @@ recipies. I found a bit better way to do it using `importlib` module
 as shown below.
 
 .. code-block:: python
+
    from __future__ import print_function
    import sys
    import importlib
@@ -52,7 +53,6 @@ python files lying around in a directory, which I wanted to plug into
 the bot during run time and use them depending on some conditions. So
 basic structure which I was looking was as follows.
 
-::
    pluginloader.py
    plugins
      |
@@ -65,6 +65,7 @@ basic structure which I was looking was as follows.
 shown below.
 
 .. code-block:: python
+
    import os
    import sys
    import re

diff --git a/dyanmic_module_loading.rst b/dyanmic_module_loading.rst
@@ -0,0 +1,103 @@
+Loading python modules in run-time
+##################################
+
+:date: 2014-04-07 12:30
+:slug: dynamic-module-loading
+:tags: python, modules, import, programming
+:author: copyninja
+:summary: Post describes about loading arbitrary python files or
+          modules during runtime.
+
+Some times it is desired to load arbitrary python files or pre
+installed python modules during application run time.I had encountered
+2 such usecases, one is in `SILPA application <http://silpa.org.in>`_ and other
+is `dictionary-bot <https://github.com/copyninja/dictionary-bot>`_
+which I was refactoring recently.
+
+Case 1: Loading installed python module
+---------------------------------------
+
+In case of SILPA I need to load pre-installed modules and here is the
+`old code
+<https://github.com/Project-SILPA/Silpa-Flask/blob/master/core/modulehelper.py#L24>_`
+, that is a bit hacky code I copied from Active State Python
+recipies. I found a bit better way to do it using `importlib` module
+as shown below.
+
+.. code-block:: python
+   from __future__ import print_function
+   import sys
+   import importlib
+
+   def load_module(modulename):
+       mod = None
+       try:
+           mod = importlib.import_module(modulename)
+       except ImportError:
+           print("Failed to load {module}".format(module=modulename),
+		        file=sys.stderr)
+       return mod
+
+Here `importlib` itself takes care of checking if modulename is
+already loaded by checking `sys.modules[modulename]`, if loaded it
+returns that value, otherwise it loads the module and sets it to
+`sys.modules[modulename]` before returning module itself.
+
+
+Case 2: Loading python files from arbitrary location
+----------------------------------------------------
+
+In case of dictionary bot my requirement was bit different, I had some
+python files lying around in a directory, which I wanted to plug into
+the bot during run time and use them depending on some conditions. So
+basic structure which I was looking was as follows.
+
+::
+   pluginloader.py
+   plugins
+     |
+     |__ aplugin.py
+     |
+     |__ bplugin.py
+
+`pluginloader.py` is the file which needs to load python files under
+`plugins` directory. This was again done using `importlib` module as
+shown below.
+
+.. code-block:: python
+   import os
+   import sys
+   import re
+   import importlib
+
+   def load_plugins():
+       pysearchre = re.compile('.py$', re.IGNORECASE)
+       pluginfiles = filter(pysearchre.search,
+                              os.listdir(os.path.join(os.path.dirname(__file__),
+                                                    'plugins')))
+       form_module = lambda fp: '.' + os.path.splitext(fp)[0]
+       plugins = map(form_module, pluginfiles)
+       # import parent module / namespace
+       importlib.import_module('plugins')
+       modules = []
+       for plugin in plugins:
+		if not plugin.startswith('__'):
+		    modules.append(importlib.import_module(plugin, package="plugins"))
+
+       return modules
+
+Above code first searches for all python file under specified
+directory and creates a relative module name from it. For eg. file
+`aplugin.py` will become `.aplugin`. Before loading modules itself we
+will load the parent module in our case `plugins`, this is because
+*relative imports in python expects parent module to be already
+loaded*. Finally for relative imports to work with
+`importlib.import_module` we need to specify parent module name in
+`package` argument. Note that we ignore files begining with *__*, or
+specifically we don't want to import __init__.py, this will be done
+when we import parent module.
+
+The above code was inspired from a `answer on StackOverflow
+<https://stackoverflow.com/a/3381582>`_, which uses `imp` module, I
+avoided `imp` because its been deprecated from *Python 3.4* in favor
+of `importlib` module.