Common Requirements Enumeration Application

This is work in progress. See the application working at https://www.opencre.org CRE is an interactive content linking platform for uniting security standards and guidelines. It offers easy and robust access to relevant information when designing, developing, testing and procuring secure software. This python web and cli application handles adding and presenting CREs.

WHY?

Independent software security professionals got together to find a solution for the complexity and fragmentation in today’s landscape of security standards and guidelines. These people are Spyros Gasteratos, Elie Saad, Rob van der Veer and friends, in close collaboration with the SKF, OpenSSF and Owasp Top 10 project.

HOW?

The CRE links each section of a standard to a shared topic (a Common Requirement), causing that section to also link with all other resources that map to the same topic. This 1) enables users to find all combined information from relevant sources, 2) it facilitates a shared and better understanding of cyber security, and 3) it allows standard makers to have links that keep working and offer all the information that readers need, so they don’t have to cover it all themselves. The CRE maintains itself: topic links in the standard text are scanned automatically. Furthermore, topics are linked with related other topics, creating a semantic web for security.

Example: the session time-out topic will take the user to relevant criteria in several standards, and to testing guides, development tips, more technical detail, threat descriptions, articles etc. From there, the user can navigate to resources about session management in general. WHEN?

CRE is currently in beta and has linked OWASP standards (Top 10, ASVS, Proactive Controls, Cheat sheets, Testing guide), plus several other sources (CWE, NIST-800 53, NIST-800 63b), as part of the OWASP Integration standard project.

Data has been kindly contributed by the SKF and ASVS projects

Installing

To install this application you need python3, yarn and virtualenv. Clone the repository:

git clone https://github.com/OWASP/common-requirement-enumeration 

Copy sqlite database to required location

cp cres/db.sqlite standards_cache.sqlite

Install dependencies

 make install 

Running

To run the CLI application, you can run

python cre.py --help

To download a remote cre spreadsheet locally you can run

python cre.py --review --from_spreadsheet < google sheets url>

To add a remote spreadsheet to your local database you can run

python cre.py --add --from_spreadsheet < google sheets url>

To run the web application for development you can run

make dev-run

Alternatively, you can use the dockerfile with

make docker && make docker-run

To run the web application for production you need gunicorn and you can run from within the cre_sync dir

make prod-run

Developing

You can run backend tests with

make test

You can run get a coverage report with

make cover

Try to keep the coverage above 70%

Repo Moved here from https://github.com/northdpole/www-project-integration-standards

Contributing

Please see Contributing for contributing instructions

Development Notes

• add tests

• defs

• db

• parsers

• mapping_add ( done for important methods ) argparse logic only remains

• spreadsheet_utils

• frontend

• add parse from export format

• add parse from export format where the root doc is a standard and it links to cres or groups

• add parse from spreadsheet with unknown standards (for key,val in items add_standard)

• merge spreadsheet to yaml and mapping add, they do the same thing

• add the ability for standards to link other standards, then you can handle assigning CREs yourself

• support importing yaml export files of more than 1 levels deep

• add export for Standards unmapped to CREs as lone standards (useful for visibility)

• add sparse_spreadsheet_export functionality one level of mapping per row, either everything that maps to standard X or everything that maps to CRE x

• add parse from export format

• add github actions ci

• make into flask rest api

• > refer use case (search by cre)

• > search by standard

• add the ability for a mapping document to have multiple yamls in it

• add db integration of tags

• add tags in db (search by tag, export with tags etc)

• add parser integration of tags (parse the new new new spreadsheet template which incorporates tags)

• add search by tag in rest

• add dockerfile

• add conditional export (select the standards you want exported get mappings between them) (gap analysis use case) ~ -- Done

• add flask cover command from here https://github.com/miguelgrinberg/flasky/blob/master/flasky.py#L33

• Make Standards versioned ~ -- Done

• write frontend

• make results per page a config item from env

• migrate to new repo

• add black autoformater

• merge frontend changes to master

• Typed Python?

= Future Considerations =

• improve test coverage -- we are at 73%, let's increase to 80%

• Make frontend show gap analysis

• Make frontend export search results and gap analysis to spreadsheet (supply backend with an "export=True" arg)

• Make frontned able to import from spreadsheet template.

• Make frontend able to import from files

• Make frontend able to import by filing in a form.

• make pagination also for tag results and gap analysis

• make library out of file format and spreadsheet template parsers

• add more linkTypes, Child, Controls, Tests, others.

• Add more Document types, Tool, Library

• Figure a way to dynamically register new Custom Resource Definitions and register custom logic on what to do on import/export and search.

• write docs and record usage gif