Some people might say, well, don’t you just do poetry publish on your project? Well, that creates a package, not necessarily a well made library. A python library has a few things
- Clean dependencies
- Heavy typing
- OOPS (Object oriented programming structure)
- Formatted code
- Open source (So others can help)
- Good documentation (Readthedocs + sphinx + RST)
This guide/tutorial will hopefully help you do all of these.
Clean dependencies
In case you do not know, the default python template for replit comes with some usually unnecessary packages. The first step is removing the upper limit on the python version, as that usually causes trouble, and removing all existing packages. (In pyproject.toml)
Before:
[tool.poetry.dependencies]
python = ">=3.10.0,<3.11"
numpy = "^1.22.2"
replit = "^3.2.4"
Flask = "^2.2.0"
urllib3 = "^1.26.12"
After:
[tool.poetry.dependencies]
python = "^3.10.0" # ^ means between 3.10.0 and 4.0.0
After that run pip install poetry -U, and run poetry add <package name> for all required packages. If you do this properly, your package should be ready to go and easy to use as soon as you publish it!
Note: If the package is not required for use, but it helpful for developing, eg. mypy or black formatter, install it using poetry add -D <package name>
Heavy typing
In Python, typing is not required, but we use type-hinting to make our code more easy to read and understand. For me, I go by this standard: Type all functions, slot and type slots when possible, and only type variables when mypy complains.
For example, take this class:
class Player:
def __init__(self, name):
self.name = name
def hello(self):
return f"Hello {self.name}"
We would type it as this:
class Player:
name: str
__slots__ = ("name") # The only attributes that can be set
def __init__(self, name: str) -> None: # Returns none
self.name = name
def hello(self) -> str: # Returns string
return f"Hello {self.name}"
We use use : to declare what kind of object is supposed to go into a parameter, and -> for what kind is supposed to be returned. __slots__ is not necessarily typing, but is grouped with it because it is for non-required and good practice. In this case, slots improves efficiency and makes your class easier to use.
You can use mypy by doing this:
- Run
poetry add -D mypy - Run
python -m mypy .
That will check your typing.
OOPS (Object oriented programming structure)
In all, there should be more classes top-level than functions or variables. If you already know OOP, this should be quite easy, if you don’t, there are a lot of good tutorials out there!
Here are some tips for OOP:
- Use special functions to make the user experience better, eg.
__getitem__,__len__ - Don’t just use classes, if you have a common structure, make sure that they are all subclasses of a base.
- If your subclasses are the only classes intended to be used, use abc with the base class
Formatted code
In a large project code can get cluttered really easily, so people made code formatters. For python, you can start formatting your code in 2 easy steps:
poetry add -D blackblack .
That will clean up all your python code, just remember to do black . every so often when your code feels cluttered again.
Open source
Open sourcing on replit is more difficult than normal, as if you just create a repo and push all the changes, you will be left with a cluttered mess. The steps should be as follows
- Create a
.gitignorewith the following content
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock
# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
*.replit
*.breakpoints
**/*.nix
pylint.txt
*.config
- Connect your replit account to GitHub
- Create a new repo from the replit ui
- Your GitHub repo should be created with a “initial commit”
- After some significant change, remember to always commit and push changes. (There are buttons)
Good documentation
This part will tell you how to setup automated documentation using sphinx+rtd+RST. For this part you will need a GitHub repo.
Part 1: Autodoc setup
- Create the file structure below
docs/
- src/
- conf.py
Makefile
make.bat
- In makefile put this:
# Minimal makefile for Sphinx documentation
# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = src
BUILDDIR = build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
- In make.bat put this:
@ECHO OFF
pushd %~dp0
REM Command file for Sphinx documentation
if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=src
set BUILDDIR=build
if "%1" == "" goto help
%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
goto end
:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
:end
popd
- And in
conf.pyput this: (Adapted to your project)
# Configuration file for the Sphinx documentation builder.
# -- Project information
project = "<your project name>"
copyright = "<year>, <name>"
author = "<name>"
release = "0.1"
version = "0.1.0"
# -- General configuration
extensions = [
"sphinx.ext.duration",
"sphinx.ext.doctest",
"sphinx.ext.autodoc",
"sphinx.ext.autosummary",
"sphinx.ext.intersphinx",
]
intersphinx_mapping = {
"python": ("https://docs.python.org/3/", None),
"sphinx": ("https://www.sphinx-doc.org/en/master/", None),
}
intersphinx_disabled_domains = ["std"]
templates_path = ["_templates"]
# -- Options for HTML output
html_theme = "sphinx_rtd_theme"
# -- Options for EPUB output
epub_show_urls = "footnote"
- That is all the basic setup, now time for the docs! Create a
index.rstfile indocs/src/. Inside that you should have something like this:
======================
My Super Cool Library
======================
A super cool library made by me!
Contents
==========
.. toctree::
api
- As you saw, we have a toctree, otherwise known as the table of contents. This tree contains the “api” file, which is where our automated documentation will go. Start by creating a
api.rstin the same directory. Start by putting this:
API
=====
.. toctree::
:maxdepth: 10
- After that, for each module in your library, create a .rst file in
docs/src/api/and add to the api toctree. For example, if I had a “player” module, I would create aplayer.rst, and addapi/playerto the toctree inapi.rst. - In each of our files, put the below, except adapted for the module
<Module name>
==============
.. automodule:: <package name>.<module name>
:members:
:undoc-members:
:show-inheritance:
For our player case it would be this:
Player
==============
.. automodule:: mypackage.player
:members:
:undoc-members:
:show-inheritance:
Part 2: hosting and further documentation
To host these new docs you made, follow these steps:
- For our docs to actually work, we need to setup some things so that it can install our lib. First, create a
.readthedocs.yamlin the home directory of your repl, and put this inside:
version: 2
build:
os: "ubuntu-20.04"
tools:
python: "3.11"
# Build from the docs/ directory with Sphinx
sphinx:
configuration: docs/src/conf.py
# Explicitly set the version of Python and its requirements
python:
install:
- requirements: docs/src/requirements.txt
- After that, create
docs/src/requirements.txtand put the below inside (adapted for your repo)
git+<your github project url>
- Push and commit changes to your GitHub repository
- Create a readthedocs account here
- Click “import a project” and add your repo
- Your docs should be building, and they will appear soon!
Things to note:
- For tutorials in the docs, you can always add plain rst files
- You can use RST in doc comments to document things for the automated docs:
class Player
"""This will **appear** in the docs!"""
name: str #: So will this!
def hello(self) -> string:
"""This will ``too``!"""
pass
- Here are the docs for docs, aka sphinx docs: Welcome — Sphinx documentation
- You can choose a theme here: https://sphinx-themes.org/
Publishing your package
To publish your package:
- Add two secrets,
PYPI_PASSWORDwith your pypi password andPYPI_USERNAMEwith your pypi username - Run this command:
poetry publish --build --password $PYPI_PASSWORD --username $PYPI_USERNAME - When releasing new version, remember to change the version in
pyproject.tomland if you have docs inconf.py
Conclusion
I hope people found this useful! Please ask questions below, and if you feel it can be improved, it is a wiki, so edit it yourself!
PS: comment if you actually read it all XD
this resource is really helpful (though I didn’t understand everything, lol). Great job dragonhunter!