Commit 657cd9fe authored by Samuel Maier's avatar Samuel Maier
Browse files

Update snapshot

parent 85b385aa
...@@ -8,6 +8,23 @@ Most of the subdirectories have their own `README.md`. ...@@ -8,6 +8,23 @@ Most of the subdirectories have their own `README.md`.
One thing that almost all directories have in common is their general [Python toolchain](#common-python-toolchain). One thing that almost all directories have in common is their general [Python toolchain](#common-python-toolchain).
## Overview over the subdirectories
* [common_py](./common_py/) contains some utilities that have been used across projects.
* [raw_data](./raw_data/) contains a bunch of gitignores ignoring non-public data and READMEs explaining what data should be there. This way, other subdirectories can reference it by path.
* [enem_aggregate](./enem_aggregate/) contains the code that converts the raw data from enem, along with the info from Marinho, into a much simpler unified CSV containing the aggregated interactions.
* [moodle_extract](./moodle_extract/) contains the code that parsed the moodle question xml and bundled it together with the experimental data in the form of a CSV.
* [qde_model_code](./qde_model_code/) contains the heart of the thing, the code that trained the ml model on the data generated by the other projects.
* [data_explore](./data_explore) (may not be added) contains some (not all) of my Python scripts that were used to gain insight into the data and often to plot the data.
### Configuration
None of the projects here have a defined CLI.
Configuration is done in the source files themselves.
I did not want to add a dependency or the complexity of having a CLI (which does not suck), and frankly with how fast the underlying code and thus the API changes, it would have been a lot of work that would have just hindered experimentation.
An effort was made to have the regularly changing configuration of each potential entry point in few places, and not spread over the repository.
## Common Python toolchain ## Common Python toolchain
The project uses Python 3.10+, and `python-poetry` (`1.5.1`) has been used for other Python package dependencies and tools. The project uses Python 3.10+, and `python-poetry` (`1.5.1`) has been used for other Python package dependencies and tools.
...@@ -73,23 +90,6 @@ Python does not have any syntactic sugar to make these easier to handle (such as ...@@ -73,23 +90,6 @@ Python does not have any syntactic sugar to make these easier to handle (such as
Also usually the "wrong" code will lead to the same results a non-null assertion would lead to in other languages, an exception saying "that thing is None!. Also usually the "wrong" code will lead to the same results a non-null assertion would lead to in other languages, an exception saying "that thing is None!.
Sadly I'm not aware of a way to tell mypy that this is fine, so I'll ignore these kinds of errors. Sadly I'm not aware of a way to tell mypy that this is fine, so I'll ignore these kinds of errors.
## Overview over the subdirectories
* [common_py](./common_py/) contains some utilities that have been used across projects.
* [raw_data](./raw_data/) contains a bunch of gitignores ignoring non-public data and READMEs explaining what data should be there. This way, other subdirectories can reference it by path.
* [enem_aggregate](./enem_aggregate/) contains the code that converts the raw data from enem, along with the info from Marinho, into a much simpler unified CSV containing the aggregated interactions.
* [moodle_extract](./moodle_extract/) contains the code that parsed the moodle question xml and bundled it together with the experimental data in the form of a CSV.
* [qde_model_code](./qde_model_code/) contains the heart of the thing, the code that trained the ml model on the data generated by the other projects.
* [data_explore](./data_explore) (may not be added) contains some (not all) of my Python scripts that were used to gain insight into the data and often to plot the data.
### Configuration
None of the projects here have a defined CLI.
Configuration is done in the source files themselves.
I did not want to add a dependency or the complexity of having a CLI (which does not suck), and frankly with how fast the underlying code and thus the API changes, it would have been a lot of work that would have just hindered experimentation.
An effort was made to have the regularly changing configuration of each potential entry point in few places, and not spread over the repository.
### Why a Monorepo, why separate projects ### Why a Monorepo, why separate projects
This was created as a monorepo from the individual projects. This was created as a monorepo from the individual projects.
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment