Developer guidelines¶
Developers and contributors of FEASST may consider the following guidelines in order to simplify collaboration.
Branch policies¶
Branches in Git are extremely useful for working on different features within the same code base and then merging them into a final product or release.
https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging
Using and naming branches in a way that is consistent may simplify this process. The policies below are based loosely on the following blog.
http://nvie.com/posts/a-successful-git-branching-model/
Main branch¶
This is the release branch. This branch may only merge with develop and hotfix branches.
GTEST: Unittest C++ and all tests pass
GCOV and LCOV: Test coverage which must be >85%
SWIG and Python unittested
Document new features
Tag
git tag -a <version number>
andgit push --tags
Develop branch¶
This is the development branch.
Tested and complete implementation of new features are added here.
Code must compile and gtests pass successfully
Consider feature compatibility with swig python interface
When merging into this branch, use –squash to add the entire feature with a single commit. Alternatively, you could also rebase your private branch.
Hotfix branch¶
The hotfix branch is for implementation of bug fixes to the main branch. After merging with main and develop, they are deleted.
Feature branches¶
These branches are for the development of new features. These branch names must begin with characters which identify the main developer of the feature (e..g, hwh/feature) The main developer sets the branch policy.
Dead (dead_*) branches¶
These branches may record an incomplete attempt of a feature that may be relevant in the future. These branch names must begin with the characters “dead”.
No rules. Code may not compile. HINT: rename branches with “git branch -m <newname>”
Pull requests¶
To create a pull request, fork the usnistgov repo, create a new branch with your changes, and add the pull request, as detailed in https://opensource.com/article/19/7/create-pull-request-github
To incorporate the pull request into feasst - git fetch usnistgov pull/ID/head:BRANCHNAME - git checkout BRANCHNAME - [make local changes] - git push usnistgov BRANCHNAME
Tools¶
GTEST: Unittest C++¶
The main branch requires GTEST coverage for all cpp plugins in plugin/name/test.
cmake -DUSE_GTEST=ON ..
make unittest -j12
./bin/unittest
use
--gtest_filter=*Name*
to run specific testsuse
./bin/unittest \|& more
to pipe stderr to stdout.use
--gtest_shuffle
to randomize the order of the testsuse
--gtest_random_seed=SEED
to reproduce an specific order.
GDB or LLDB: Debugging¶
gdb (or lldb on macOS) is especially useful for identifying segfaults via backtraces. The -g flag in compilation pulls the symbols so that you can get correct line numbers in the gdb output.
In bash
gdb [program executable name]
r [flags]
or
gdb --batch --command=../dev/test.gdb ./bin/unittest
gdb can also be used with python as
export PYTHONPATH=$PYTHONPATH:~/feasst/build/
gdb python
r [python script] [optional flags]
use ‘gdb catch throw’ or ‘lldb break set -E C++’ to backtrace exceptions
use gdb as a profiler by ctrl c in the middle and backtrace: https://stackoverflow.com/a/378024
use gdb as a parallel profiler: http://poormansprofiler.org/
Valgrind: Find memory leaks¶
Valgrind helps to detect memory management bugs.
For example, to run Valgrind on a particular test and output to text file
valgrind ./unittest --gtest_filter=MC.* > out.txt 2>&1
For uninitialized value errors, try –track-origins=yes
For leaks, try –leak-check=full –show-leak-kinds=all
Don’t use profiler for leak checks. OMP causes “leaks” O.K.
For suppress false-positives (e.g., gomp or gsl), use –gen-suppressions=all to generate suppression files
GCOV and LCOV: Test coverage¶
GCC compilers allow testing of coverage with gcov and lcov for visualization.
Code: currently implemented with Travis CI and CodeCov and available online. See .travis.yml for example of how to use lcov
Use GCOV with CMake: cmake -DUSE_GCOV . Note: this disables optimization, so don’t use it for production simulations.
make coverage
Open coverage/index.html in your browser.
Go into “src” and ignore the external library coverage.
CCACHE: Speed up compilation time¶
Something as trivial as changing a comment in a header file can lead to a massive recompile of the entire source. Your previous compile is remembered by ccache, leading to near instant recompilation in the above example.
cpplint and pylint: Clean up styling¶
Document¶
Setup¶
pip install sphinx breathe doxygen with GENERATE_XML run sphinx-quickstart, enable autodoc add something like the following to your sphinx index.rst:
.. doxygenclass:: Nutshell
:project: nutshell
:members:
- add the following to your sphinx conf.py
extensions = [ “breathe”, “nbsphinx” ] breathe_projects = {“FEASST”:”../xml”} breathe_domain_by_extension = {“h” : “cc”}
pip install sphinx_rtd_theme nbsphinx
run sphinx: make html
Sphinx/Breathe/Doxygen notes¶
Link from rst file to C++ function:
:cpp:func:`link <feasst::className::function()>`
Link from rst file to C++ class:
:cpp:class:`link <feasst::className>`
Link from rst file to fst file:
:doc:`/tutorial/asdf`
[note, / references root]Link from rst file to ipynb file :
`Tutorial <tutorial/tutorial.html>`_
Link from C++ to C++:
className::function()
Link from C++ source to rst file:
<a href="tutorial/asdf.html">test</a>
For math in C++ comments:
\f$ latex code here \f$
For tables, see monte_carlo/include/trial_compute_add.h
Pip notes¶
dev/tools/pip_install.sh
Anaconda cloud notes¶
https://docs.anaconda.com/anaconda-cloud/user-guide/getting-started/ * conda build purge * still haven’t gotten this to work because of overlinking
Style¶
Reference guides for C++¶
Naming¶
ClassNames are mixed case with starting upper case letter
member_names are lower case with underscores
private_member_names_ end with an underscore
function_names are also lower case with underscores
bools syntax: is_[accepted.., etc]
MACROS and CONSTANTS are all upper case.
Avoid MACROS and CONSTANTS.
use “and”, “or” instead of “&&”, “||” (HWH: change this to follow Google?)
Functions¶
Use return values. Argument ordering: input (value or constant reference), then output (pointer only)
Overloaded functions -> can you document all in a single comment? good
No Default parameters on virtual functions
Classes¶
Nearly all data members should be private. Limit protected members
member_name() returns const member
set_member_name(member_name) sets member
For setters with multiple arguments, the first are vector indices as in order x[0] = 3…
getptr_member_name returns constant pointer (optimization only)
Loops and if¶
use of “for (auto element : container) { … }” is dangerous
for simple loops over containers, use “for (element : container)”
for loops where you need the index, use: for (int index = 0; index < static_cast<int>(container.size()); ++index)
Auto¶
only use auto when the type is clear such as auto var = std::make_shared<..>.
Arguments¶
All arguments are provided as strings and converted to the expected type.
Check that all arguments are used (e.g., like implicit none, a typo is caught).
Argument defaults need to be set and clearly commented.
If no default, it is a required argument.
Serialization¶
For inheritance hierarchy, a static deserialize_map is used to relate class name to template.
Each object serializes a version that can be used for checks and backwards compatibility.
utils_io.h contains many function templates for serialization.
In particular, feasst_deserialize_fstdr() needs to be fixed.
Don’t forget to serialize (private) member data in new implementations.
To compare differences between two serializations, paste into file and using “s/ /r/g”
File output¶
comma-separated values (CSV) are the preferred format (e.g., comma deliminter)
For quick reference¶
line counts [find . -name ‘.cpp’ -o -name ‘.h’ | xargs wc -l | sort -n]
tutorial errors [ find . -name ‘tutorial_failures.txt’ | xargs cat ]
clean docs before running depend.py again [ for dir in ls –color=never -d *; do rm $dir/doc/*rst; done ]
find difference in serialization string: [ diff -u f1 f2 |colordiff | perl /usr/share/doc/git/contrib/diff-highlight/diff-highlight | more ]
To Do List¶
implement gibbs ensemble
find a better way for two different classes to take from the same argument list and still maintain unused checks.
Make utils:lj,spce,etc derived classes of System ?
benchmark feasst vs simple hardcoded LJ simulations. Create benchmarking profile to compare among versions
ideal gas as the first tutorial/testcase
specify units in LMP data files?
fix dependency linkers required by clang/cmake on macOS but not g++ on ubuntu
consider optimization of Ewald: init ewald storage on particle types, precompute property index.
when selecting from cpdf, use lnp instead of p?
insert optimization: update cell list of sites when added, but of domain only when finalized.
IF using argtype for custom object, considering single string constructors. E.g., for position in cylinder.h, use {“point0”, “0 0 0”}
Python debug script: easy for user to run gdb/valgrind
Toggle more debug levels, and localized to certain files/plugins, etc
force precompute when reinitializing system, criteria, etc in MonteCarlo
MonteCarlo subclass Simulation
swig python wrap arguments std::istream and std::ostream for serialization
add citations to tutorials (reweighting, etc) and also citation suggestions for MC objects
VisitModels may prefer to update select properties (e.g., cell, eik)
Jupyter notebook output should go to cells, not terminal that runs jupyter.
lint file_[xyz,lmp]
regrow but within near existing, for ‘free dof, e.g. azimuthal in angle, sphere in bond, etc’
put cell list in finalize-heavy paradigm, update_positions updates cell of selection, finalize updates entire cell list. linked list
config could use revert,finalize to update cell list only on finalization, and maybe not have to exclude from cell properties (why exclude?). same with ewald
Refactor arguments so that they can be checked for usage (especially in Trials)
Rename TrialSelect->SelectTrial, TrialCompute->ComputeTrial. Rename Compute->Decide?.
Somehow, trial_growth_expanded.h doesn’t include debug.h but can compile with ASSERT
Speed up RNG by maintaining int_distribution like dis_double
Document utils lj, spce, rpm in tutorials
Add a FAQ for common compile errors: “no known conversion from brace-enclosed initializer list to const argtype&” often means that a parameter was not converted to a string.
Make a CachedRandom and CachedPotential for prefetch and avoid if statements that could slow down serial simulations.
remove tutorial/fh.py
Tuner->Tune
Analyze/ModifyFactory optimization: use steps_per in factory to limit number of checks
implement timer for profiles (with hierarchies by class… tried this, but its too slow. Time only infrequently?)
implement a timer to auto-balance trial weights based on cpu time.
More documentation/tutorial on how to create your own plugins, classes, etc
add orientation argument to shapes with internal coordinate transformation
System should track current energy of every potential for analysis (Criteria running energies may contain a part from each potential to simplify debugging).
Consider using new state instead of old state in acceptance derivations
Sort selection_of_all, or impose sorting in Select::add_particles. Currently, this leads to issues.
Rename data and xyz files, document them more cleary (second line in xyz, and error if data not read correctly).
Make ModelTwoBodyTable that tabulates interaction from min(hs)-max(rc) distance for each distinct pair of site types, and can easily be added as optimized Potential
Rename plugin chain->config_bias ?
maybe mc.add(criteria) is preferable to mc.set. Same with sys?
in optimizing where config only updates when trial finalized, how to build off new perturbed config in CB?