{ "cells": [ { "cell_type": "markdown", "id": "5fb0b079", "metadata": {}, "source": [ "# docfiller\n", "\n", ":::{eval-rst}\n", ".. currentmodule:: module_utilities.docfiller\n", ":::\n", "\n", "\n", ":::{note}\n", "{mod}`~module_utilities.docfiller` assumes Numpy style docstrings\n", ":::\n", "\n", "\n", "It is common to want to share documentation information across functions/methods. The module {mod}`~module_utilities.docfiller` provides a way to do this:\n", "\n", "For example, if you have these function:" ] }, { "cell_type": "code", "execution_count": 1, "id": "d08cf223", "metadata": {}, "outputs": [], "source": [ "def a_function(x, y, z):\n", " \"\"\"\n", " A function.\n", "\n", " A longer description\n", "\n", "\n", " Parameters\n", " ----------\n", " x : float\n", " An x param\n", " y : int\n", " A y param\n", " z : str\n", " A z param\n", "\n", "\n", " Returns\n", " -------\n", " output : float\n", " An output\n", " \"\"\"\n", "\n", "\n", "def b_function(x, y, z):\n", " \"\"\"\n", " A different function.\n", "\n", " A longer description of b_function\n", "\n", "\n", " Parameters\n", " ----------\n", " x : float\n", " An x param\n", " y : float\n", " A different y param\n", " z : str\n", " A z param\n", "\n", "\n", " Returns\n", " -------\n", " output : float\n", " An different output\n", " \"\"\"" ] }, { "cell_type": "markdown", "id": "8a130f6b", "metadata": {}, "source": [ "It would be nice to share info. {class}`DocFiller` provides a few ways to do this. \n", "\n", "First, we can define common docstrings like so:" ] }, { "cell_type": "code", "execution_count": 2, "id": "55e4aafe", "metadata": {}, "outputs": [], "source": [ "from module_utilities.docfiller import DocFiller" ] }, { "cell_type": "code", "execution_count": 3, "id": "b09fcf77", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'summary': '',\n", " 'extended_summary': '',\n", " 'parameters': {'x': 'x : float\\n An x param',\n", " 'z': 'z : str\\n A z param'},\n", " 'returns': '',\n", " 'yields': '',\n", " 'notes': '',\n", " 'warnings': '',\n", " 'other_parameters': '',\n", " 'attributes': '',\n", " 'methods': '',\n", " 'references': '',\n", " 'examples': '',\n", " 'x': 'x : float\\n An x param',\n", " 'z': 'z : str\\n A z param'}" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "docstrings = \"\"\"\n", "Parameters\n", "----------\n", "x : float\n", " An x param\n", "z : str\n", " A z param\n", "\"\"\"\n", "\n", "d = DocFiller.from_docstring(docstrings, combine_keys=\"parameters\")\n", "\n", "d.data" ] }, { "cell_type": "markdown", "id": "5565b624", "metadata": {}, "source": [ "This parses the `docstring` (using numpydoc). Then, we can define functions as:" ] }, { "cell_type": "code", "execution_count": 4, "id": "378cea23", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "A function.\n", "\n", "A longer description\n", "\n", "\n", "Parameters\n", "----------\n", "x : float\n", " An x param\n", "y : int\n", " A y param\n", "z : str\n", " A z param\n", "\n", "\n", "Returns\n", "-------\n", "output : float\n", " An output\n", "\n", "\n", "A different function.\n", "\n", "A longer description of b_function\n", "\n", "\n", "Parameters\n", "----------\n", "x : float\n", " An x param\n", "y : float\n", " A different y param\n", "z : str\n", " A z param\n", "\n", "\n", "Returns\n", "-------\n", "output : float\n", " An different output\n", "\n" ] } ], "source": [ "from textwrap import dedent\n", "\n", "\n", "@d()\n", "def a_function2(x, y, z):\n", " \"\"\"\n", " A function.\n", "\n", " A longer description\n", "\n", "\n", " Parameters\n", " ----------\n", " {x}\n", " y : int\n", " A y param\n", " {z}\n", "\n", "\n", " Returns\n", " -------\n", " output : float\n", " An output\n", " \"\"\"\n", "\n", "\n", "@d()\n", "def b_function2(x, y, z):\n", " \"\"\"\n", " A different function.\n", "\n", " A longer description of b_function\n", "\n", "\n", " Parameters\n", " ----------\n", " {x}\n", " y : float\n", " A different y param\n", " {z}\n", "\n", "\n", " Returns\n", " -------\n", " output : float\n", " An different output\n", " \"\"\"\n", "\n", "\n", "assert dedent(a_function.__doc__) == a_function2.__doc__\n", "assert dedent(b_function.__doc__) == b_function2.__doc__\n", "\n", "print(a_function2.__doc__)\n", "print(b_function2.__doc__)" ] }, { "cell_type": "markdown", "id": "e79cceae", "metadata": {}, "source": [ "## Using templates" ] }, { "cell_type": "markdown", "id": "a3edc3c3", "metadata": {}, "source": [ "Better still is to generalize the template:" ] }, { "cell_type": "code", "execution_count": 5, "id": "95067163", "metadata": {}, "outputs": [], "source": [ "@ (\n", " d.update(\n", " summary=\"A function.\",\n", " extended_summary=\"\"\"\n", " A longer description\n", " \"\"\",\n", " )\n", " .assign_param(\"y\", \"int\", \"A y param\")\n", " .assign_param(\"output\", \"float\", \"An output\")\n", ").dedent().decorate\n", "def a_function3(x, y, z):\n", " \"\"\"\n", " {summary}\n", "\n", " {extended_summary}\n", "\n", "\n", " Parameters\n", " ----------\n", " {x}\n", " {y}\n", " {z}\n", "\n", "\n", " Returns\n", " -------\n", " {output}\n", " \"\"\"\n", "\n", "\n", "assert a_function3.__doc__ == a_function2.__doc__\n", "\n", "\n", "@ (\n", " d.update(\n", " summary=\"A different function.\",\n", " extended_summary=\"A longer description of b_function\",\n", " )\n", " .assign_param(\"y\", \"float\", \"A different y param\")\n", " .assign_param(\"output\", \"float\", \"An different output\")\n", ").dedent()(a_function3)\n", "def b_function3(x, y, z):\n", " pass\n", "\n", "\n", "assert b_function3.__doc__ == b_function2.__doc__" ] }, { "cell_type": "markdown", "id": "4fd7eb47", "metadata": {}, "source": [ "Note that for `b_function3`, we used the template/docstring from `a_function3`. {class}`DocFiller` defaults to using\n", "the functions docstring for replacement. But you can pass a string or function to use as the template to be filled. Above, we also used {meth}`DocFiller.assign_param` to create a correctly formatted parameter." ] }, { "cell_type": "markdown", "id": "6fcd8202", "metadata": {}, "source": [ "## General shared docs\n", "\n", "Another approach is to define shared parameters, with slight variations in there own Docfiller instance, and use this\n", "for replacement." ] }, { "cell_type": "code", "execution_count": 6, "id": "ac0be077", "metadata": {}, "outputs": [], "source": [ "da = DocFiller.from_docstring(\n", " \"\"\"\n", " A function.\n", "\n", " A longer description\n", "\n", " Parameters\n", " ----------\n", " y : int\n", " A y param\n", "\n", " Returns\n", " -------\n", " output : float\n", " An output\n", " \"\"\",\n", " # this makes parameters and returns available at top level.\n", " # otherwise, have to access using `parameters.y`, etc.\n", " combine_keys=[\"parameters\", \"returns\"],\n", ")\n", "\n", "db = DocFiller.from_docstring(\n", " \"\"\"\n", " A different function.\n", "\n", " A longer description of b_function\n", "\n", " Parameters\n", " ----------\n", " y : float\n", " A different y param\n", "\n", " Returns\n", " -------\n", " output : float\n", " An different output\n", " \"\"\",\n", " combine_keys=[\"parameters\", \"returns\"],\n", ")" ] }, { "cell_type": "code", "execution_count": 7, "id": "0bf506c0", "metadata": {}, "outputs": [], "source": [ "@d.append(da)(a_function3)\n", "def a_function4():\n", " pass\n", "\n", "\n", "assert a_function4.__doc__ == a_function2.__doc__\n", "\n", "\n", "@d.append(db)(a_function3)\n", "def b_function4():\n", " pass\n", "\n", "\n", "assert b_function4.__doc__ == b_function4.__doc__" ] }, { "cell_type": "markdown", "id": "b9f98ace", "metadata": {}, "source": [ "## Define parameters/returns that have different key values\n", "\n", "You may find that you can define a host of shared parameter documentation, except that there are a few edge cases.\n", "For example, say we have a parameter `beta` that in some cases is an array, and in others is a float. You could handle this as follows" ] }, { "cell_type": "code", "execution_count": 8, "id": "581a3f3b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "A thing\n", "\n", "\n", "Parameters\n", "----------\n", "x : float\n", " x parameter\n", "y : float\n", " y parameter\n", "beta : float\n", " A float value for beta\n", "\n", "\n", "An array thing\n", "\n", "\n", "Parameters\n", "----------\n", "x : float\n", " x parameter\n", "y : float\n", " y parameter\n", "beta : array-like\n", " An array of beta values\n", "\n" ] } ], "source": [ "docstring = \"\"\"\n", "Parameters\n", "----------\n", "beta_float | beta : float\n", " A float value for beta\n", "beta_array | beta : array-like\n", " An array of beta values\n", "\n", "x : float\n", " x parameter\n", "y : float\n", " y parameter\n", "\"\"\"\n", "\n", "d = DocFiller.from_docstring(docstring, combine_keys=\"parameters\")\n", "\n", "# using `.decorate` uses the decorator without template or parameters\n", "\n", "\n", "@d.decorate\n", "def func_float(x, y, beta):\n", " \"\"\"\n", " A thing\n", "\n", "\n", " Parameters\n", " ----------\n", " {x}\n", " {y}\n", " {beta_float}\n", " \"\"\"\n", "\n", "\n", "@d.decorate\n", "def func_array(x, y, beta):\n", " \"\"\"\n", " An array thing\n", "\n", "\n", " Parameters\n", " ----------\n", " {x}\n", " {y}\n", " {beta_array}\n", " \"\"\"\n", "\n", "\n", "print(func_float.__doc__)\n", "print(func_array.__doc__)" ] }, { "cell_type": "markdown", "id": "b282bf98", "metadata": {}, "source": [ "Or, if everyting else is the same, you could do:" ] }, { "cell_type": "code", "execution_count": 9, "id": "6db66b94", "metadata": {}, "outputs": [], "source": [ "# back to call because passing parameters\n", "\n", "\n", "@d(beta=d[\"beta_float\"], summary=\"A thing\")\n", "def func_float2(x, y, beta):\n", " \"\"\"\n", " {summary}\n", "\n", "\n", " Parameters\n", " ----------\n", " {x}\n", " {y}\n", " {beta}\n", " \"\"\"\n", "\n", "\n", "@d(func_float2, beta=d[\"beta_array\"], summary=\"An array thing\")\n", "def func_array2(x, y, beta):\n", " pass\n", "\n", "\n", "assert func_float2.__doc__ == func_float.__doc__\n", "assert func_array2.__doc__ == func_array.__doc__" ] }, { "cell_type": "markdown", "id": "0a5a4821", "metadata": {}, "source": [ "Better still, you can use {meth}`DocFiller.assign_keys`:" ] }, { "cell_type": "code", "execution_count": 10, "id": "4fee935c", "metadata": {}, "outputs": [], "source": [ "@d.assign_keys(beta=\"beta_float\")(summary=\"A thing\")\n", "def func_float3(x, y, beta):\n", " \"\"\"\n", " {summary}\n", "\n", "\n", " Parameters\n", " ----------\n", " {x}\n", " {y}\n", " {beta}\n", " \"\"\"\n", "\n", "\n", "@d.assign_keys(beta=\"beta_array\")(func_float3, summary=\"An array thing\")\n", "def func_array3(x, y, beta):\n", " pass\n", "\n", "\n", "assert func_float3.__doc__ == func_float.__doc__\n", "assert func_array3.__doc__ == func_array.__doc__" ] }, { "cell_type": "code", "execution_count": 11, "id": "68fcb14e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Parameters\n", "----------\n", "x : int\n", " x param\n", "y : float\n", " y param\n", "z : int\n", " z int param\n", "\n" ] } ], "source": [ "from module_utilities.docfiller import DocFiller\n", "\n", "d = DocFiller.from_docstring(\n", " \"\"\"\n", " Parameters\n", " ----------\n", " x : int\n", " x param\n", " y : float\n", " y param\n", " z0 | z : int\n", " z int param\n", " z1 | z : float\n", " z float parma\n", " \"\"\",\n", " combine_keys=\"parameters\",\n", ")\n", "\n", "\n", "@d()\n", "def func0():\n", " \"\"\"\n", " Parameters\n", " ----------\n", " {x}\n", " {y}\n", " {z0}\n", " \"\"\"\n", "\n", "\n", "print(func0.__doc__)" ] }, { "cell_type": "markdown", "id": "7c7039c5", "metadata": {}, "source": [ "## Works with classes\n", "\n", "This also works with classes" ] }, { "cell_type": "code", "execution_count": 12, "id": "bf6b8405", "metadata": {}, "outputs": [], "source": [ "expected = \"\"\"\n", "A summary\n", "\n", "A longer summary\n", "\n", "Parameters\n", "----------\n", "x : float\n", " x param\n", " some other stuff\n", "y : float\n", " y param\n", "\n", "Returns\n", "-------\n", "out : float\n", " output\n", "\"\"\"\n", "\n", "d = DocFiller.from_docstring(expected, combine_keys=\"parameters\")" ] }, { "cell_type": "code", "execution_count": 13, "id": "2eb1580a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on class hello in module __main__:\n", "\n", "class hello(builtins.object)\n", " | A summary\n", " | \n", " | A longer summary\n", " | \n", " | Parameters\n", " | ----------\n", " | x : float\n", " | x param\n", " | some other stuff\n", " | y : float\n", " | y param\n", " | \n", " | Returns\n", " | -------\n", " | out : float\n", " | output\n", " | \n", " | Data descriptors defined here:\n", " | \n", " | __dict__\n", " | dictionary for instance variables (if defined)\n", " | \n", " | __weakref__\n", " | list of weak references to the object (if defined)\n", "\n" ] } ], "source": [ "@d()\n", "class hello:\n", " \"\"\"\n", " {summary}\n", "\n", " {extended_summary}\n", "\n", " Parameters\n", " ----------\n", " {x}\n", " {y}\n", "\n", " Returns\n", " -------\n", " {returns.out}\n", " \"\"\"\n", "\n", "\n", "assert hello.__doc__ == expected\n", "\n", "\n", "@d(hello)\n", "class hello2(hello):\n", " pass\n", "\n", "\n", "assert hello2.__doc__ == expected\n", "\n", "help(hello)" ] }, { "cell_type": "markdown", "id": "cf9afec1", "metadata": {}, "source": [ "There are cases where you'd like to update the documentation from one implementation to another. For example, say you have the following functions:" ] }, { "cell_type": "code", "execution_count": 15, "id": "57942b42", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Add numbers\n", "\n", "Parameters\n", "----------\n", "x : float\n", " x param\n", " some other stuff\n", "y : float\n", " y param\n", "\n", "Returns\n", "-------\n", "output : float\n", " Sum of input\n", "\n" ] } ], "source": [ "@d.decorate\n", "def func_xy(x, y):\n", " \"\"\"\n", " Add numbers\n", "\n", "\n", " Parameters\n", " ----------\n", " {x}\n", " {y}\n", "\n", "\n", " Returns\n", " -------\n", " output : float\n", " Sum of input\n", " \"\"\"\n", "\n", " return x + y\n", "\n", "\n", "print(func_xy.__doc__)" ] }, { "cell_type": "markdown", "id": "16db1a4c", "metadata": {}, "source": [ "But what if you want a new version that adds three numbers `func_xyz(x, y, z)`? Most everything is the same as for `func_xy` above, but we need to add `z`. For this, we have an interface to [docstring-inheritance](https://github.com/AntoineD/docstring-inheritance)" ] }, { "cell_type": "code", "execution_count": 17, "id": "d018e5ac", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Add numbers\n", "\n", "Parameters\n", "----------\n", "x : float\n", " x param\n", " some other stuff\n", "y : float\n", " y param\n", "z : float\n", " Another parameter\n", "\n", "Returns\n", "-------\n", "output : float\n", " Sum of input\n" ] } ], "source": [ "@d.inherit(func_xy)\n", "def func_xyz(x, y, z):\n", " \"\"\"\n", " Parameters\n", " ----------\n", " z : float\n", " Another parameter\n", " \"\"\"\n", " return x + y + z\n", "\n", "\n", "print(func_xyz.__doc__)" ] }, { "cell_type": "markdown", "id": "576e8513", "metadata": {}, "source": [ "For the special case that you want to inherit from methods of a baseclass, you can use the following:" ] }, { "cell_type": "code", "execution_count": 22, "id": "58e5a50f", "metadata": {}, "outputs": [], "source": [ "class Base:\n", " def prop(self):\n", " \"\"\"A property\"\"\"\n", " return 1\n", "\n", " @d.decorate\n", " def meth(self, x):\n", " \"\"\"\n", " A method\n", "\n", "\n", " Parameters\n", " ----------\n", " {x}\n", "\n", "\n", " Returns\n", " -------\n", " float\n", " Double x\n", " \"\"\"\n", "\n", " return x * 2\n", "\n", "\n", "d_from_base = d.factory_inherit_from_parent(Base)\n", "\n", "\n", "class Derived:\n", " @d_from_base()\n", " def prop(self):\n", " pass\n", "\n", " @d_from_base()\n", " def meth(self, x, y):\n", " \"\"\"\n", " Parameters\n", " ----------\n", " {y}\n", " \"\"\"\n", " return (x + y) * 2" ] }, { "cell_type": "code", "execution_count": 23, "id": "f986e10a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on class Base in module __main__:\n", "\n", "class Base(builtins.object)\n", " | Methods defined here:\n", " | \n", " | meth(self, x)\n", " | A method\n", " | \n", " | Parameters\n", " | ----------\n", " | x : float\n", " | x param\n", " | some other stuff\n", " | \n", " | Returns\n", " | -------\n", " | float\n", " | Double x\n", " | \n", " | prop(self)\n", " | A property\n", " | \n", " | ----------------------------------------------------------------------\n", " | Data descriptors defined here:\n", " | \n", " | __dict__\n", " | dictionary for instance variables (if defined)\n", " | \n", " | __weakref__\n", " | list of weak references to the object (if defined)\n", "\n" ] } ], "source": [ "help(Base)" ] }, { "cell_type": "code", "execution_count": 24, "id": "9f7eaeef", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on class Derived in module __main__:\n", "\n", "class Derived(builtins.object)\n", " | Methods defined here:\n", " | \n", " | meth(self, x, y)\n", " | A method\n", " | \n", " | Parameters\n", " | ----------\n", " | x : float\n", " | x param\n", " | some other stuff\n", " | y : float\n", " | y param\n", " | \n", " | Returns\n", " | -------\n", " | float\n", " | Double x\n", " | \n", " | prop(self)\n", " | A property\n", " | \n", " | ----------------------------------------------------------------------\n", " | Data descriptors defined here:\n", " | \n", " | __dict__\n", " | dictionary for instance variables (if defined)\n", " | \n", " | __weakref__\n", " | list of weak references to the object (if defined)\n", "\n" ] } ], "source": [ "help(Derived)" ] } ], "metadata": { "kernelspec": { "display_name": "module-utilities-dev [conda env:dev-4]", "language": "python", "name": "conda-env-dev-4-py" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 5 }