Skip to content

Commit

Permalink
Dev (#237)
Browse files Browse the repository at this point in the history
* rework in progress.

* fix: correct limits for orionoid

* fix: switch to comprehensions

* fix: disable plex logging for id mismatches

* time for sleep. rework still wip.

* feat: parser works. needs more work. language needs a rewrite. disabled for now.

* fix: overseerr bug on using external ids

* fix: remove plex debug line for users

* disable tvdb checks from listrr, overseerr, plex. needs reworked.

* set torrentio to disabled by default. removed parse logs. raised torrentio limit slightly.

* Set all default settings to disabled by default for onboarding

* add extra logging attr. for debugging large groups of data.

* feat: started status page rewrite

* add dev branch builds with :dev tag (#165)

* fix: listrr validation

* rework in progress.

* fix: correct limits for orionoid

* fix: switch to comprehensions

* fix: disable plex logging for id mismatches

* time for sleep. rework still wip.

* feat: parser works. needs more work. language needs a rewrite. disabled for now.

* feat: frontend improvements (#158)

* feat: added global debug for settings

* feat: added dev to formDebug so it's always true in development but false in production

* feat: added DEBUG & LOG to general settings

* deps: switched svelte-sonner to shadcn customized toast component

* refactor: renamed PlexDebridItem to IcebergItem and added changes made to /items

* fix: fixed the wrong relative date in status page

* docs: readme improvements

* docs: readme improvements

* docs: readme improvements

* chore(deps): bump lucide-svelte from 0.303.0 to 0.307.0 in /frontend (#124)

* refactor: componentized forms, soon will do same for fields too

* Parse rewrite (#128)

* Move parser to its own module

* Add ORIGIN to env vars

* Fix overseerr, watchlist, jackett validation.

* Added more refined logic to parser module.

* Set stage for testing

* Add methods for individual checks

* Update sort logic

* Update default settings

* Fix jackett. Begin to add title support for jackett.

---------

Co-authored-by: Spoked <Spoked@localhost>
Co-authored-by: Dreu LaVelle <dreu.lavelle@localhost>

* feat: onboarding on the way ;), major refactoring of form related code

* Simplified downloading logic and modified state matchine

* fix typo in state machine and handle movie pathing correctly

* Remove useless method

* Temporary fix to test

* Remove uncached stream hashes from item to avoid loop, some blacklisting logic could also be good

* chore(deps-dev): bump @typescript-eslint/eslint-plugin in /frontend (#134)

* chore(deps): bump lucide-svelte from 0.307.0 to 0.309.0 in /frontend (#133)

* chore(deps-dev): bump @sveltejs/kit from 2.0.1 to 2.3.2 in /frontend (#132)

* chore(deps): bump bits-ui from 0.13.0 to 0.14.0 in /frontend (#130)

* chore(deps-dev): bump @sveltejs/adapter-node in /frontend (#138)

* feat: some more onboarding and form improvements

* Dev startup to disabling pickling

* feat: Listrr Support Added (#136)

* Start Listrr Feature
* feat: Listrr ready for review.
* small tweaks. rewrite coming later.

---------

Co-authored-by: Spoked <Spoked@localhost>

* Jackett rewrite (#139)

* Add TorBox scraper

* Add is_anime attribute to item

* Rework Jackett to Keyword Queries. Added categories. Removed Torbox

* Remove audio from parsing, it removed alot of good hits

* fix movie scraping and modify response parsing logic to be more readable

* fix: remove torbox module

* remove audio from being parsed

* remove more audio from parser

* fix typo

* fix: tidy audio and networks

* small tweaks

---------

Co-authored-by: Spoked <Spoked@localhost>
Co-authored-by: Gaisberg <None>

* Avoid [None] if empty content service

* fix: handle bad quality manually in parser (#145)

Co-authored-by: Spoked <Spoked@localhost>

* deps: updated deps due to security updateS

* feat: added more onboarding steps, some bugs also introduced

* chore(deps-dev): bump @sveltejs/kit from 2.3.2 to 2.4.2 in /frontend (#156)

* chore(deps-dev): bump prettier from 3.1.1 to 3.2.4 in /frontend (#155)

* chore(deps): bump lucide-svelte from 0.309.0 to 0.314.0 in /frontend (#154)

* chore(deps): bump bits-ui from 0.14.0 to 0.15.1 in /frontend (#153)

* feat: minor changes

* feat: deps change

* feat: deps change

* feat: onboarding MVP done

---------

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Spoked <[email protected]>
Co-authored-by: Spoked <Spoked@localhost>
Co-authored-by: Dreu LaVelle <dreu.lavelle@localhost>
Co-authored-by: Gaisberg <None>

* feat: frontend improvements (#159)

* feat: added global debug for settings

* feat: added dev to formDebug so it's always true in development but false in production

* feat: added DEBUG & LOG to general settings

* deps: switched svelte-sonner to shadcn customized toast component

* refactor: renamed PlexDebridItem to IcebergItem and added changes made to /items

* fix: fixed the wrong relative date in status page

* docs: readme improvements

* docs: readme improvements

* docs: readme improvements

* chore(deps): bump lucide-svelte from 0.303.0 to 0.307.0 in /frontend (#124)

* refactor: componentized forms, soon will do same for fields too

* Parse rewrite (#128)

* Move parser to its own module

* Add ORIGIN to env vars

* Fix overseerr, watchlist, jackett validation.

* Added more refined logic to parser module.

* Set stage for testing

* Add methods for individual checks

* Update sort logic

* Update default settings

* Fix jackett. Begin to add title support for jackett.

---------

Co-authored-by: Spoked <Spoked@localhost>
Co-authored-by: Dreu LaVelle <dreu.lavelle@localhost>

* feat: onboarding on the way ;), major refactoring of form related code

* Simplified downloading logic and modified state matchine

* fix typo in state machine and handle movie pathing correctly

* Remove useless method

* Temporary fix to test

* Remove uncached stream hashes from item to avoid loop, some blacklisting logic could also be good

* chore(deps-dev): bump @typescript-eslint/eslint-plugin in /frontend (#134)

* chore(deps): bump lucide-svelte from 0.307.0 to 0.309.0 in /frontend (#133)

* chore(deps-dev): bump @sveltejs/kit from 2.0.1 to 2.3.2 in /frontend (#132)

* chore(deps): bump bits-ui from 0.13.0 to 0.14.0 in /frontend (#130)

* chore(deps-dev): bump @sveltejs/adapter-node in /frontend (#138)

* feat: some more onboarding and form improvements

* Dev startup to disabling pickling

* feat: Listrr Support Added (#136)

* Start Listrr Feature
* feat: Listrr ready for review.
* small tweaks. rewrite coming later.

---------

Co-authored-by: Spoked <Spoked@localhost>

* Jackett rewrite (#139)

* Add TorBox scraper

* Add is_anime attribute to item

* Rework Jackett to Keyword Queries. Added categories. Removed Torbox

* Remove audio from parsing, it removed alot of good hits

* fix movie scraping and modify response parsing logic to be more readable

* fix: remove torbox module

* remove audio from being parsed

* remove more audio from parser

* fix typo

* fix: tidy audio and networks

* small tweaks

---------

Co-authored-by: Spoked <Spoked@localhost>
Co-authored-by: Gaisberg <None>

* Avoid [None] if empty content service

* fix: handle bad quality manually in parser (#145)

Co-authored-by: Spoked <Spoked@localhost>

* deps: updated deps due to security updateS

* feat: added more onboarding steps, some bugs also introduced

* chore(deps-dev): bump @sveltejs/kit from 2.3.2 to 2.4.2 in /frontend (#156)

* chore(deps-dev): bump prettier from 3.1.1 to 3.2.4 in /frontend (#155)

* chore(deps): bump lucide-svelte from 0.309.0 to 0.314.0 in /frontend (#154)

* chore(deps): bump bits-ui from 0.14.0 to 0.15.1 in /frontend (#153)

* feat: minor changes

* feat: deps change

* feat: deps change

* feat: onboarding MVP done

* refactor: moved schemes into forms/helpers.ts and command menu improvements

* refactor: switched to new font, changes made to all except status page

* refactor: minor change, didn't get commit

* fix: minor fix

* feat: fixed git merge conflicts issue

* feat: fixed status page font too

---------

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Spoked <[email protected]>
Co-authored-by: Spoked <Spoked@localhost>
Co-authored-by: Dreu LaVelle <dreu.lavelle@localhost>
Co-authored-by: Gaisberg <None>

* docs: minor improvements (#160)

* docs: readme improvements

* docs: readme improvements

* docs: minor improvements (#161)

* docs: readme improvements

* docs: readme improvements

* docs: readme improvements

* docs: minor improvements (#162)

* fix: correct parsing of external id's (#163)

Co-authored-by: Dreu LaVelle <[email protected]>

* fix: overseerr bug on using external ids

* fix: remove plex debug line for users

* disable tvdb checks from listrr, overseerr, plex. needs reworked.

* set torrentio to disabled by default. removed parse logs. raised torrentio limit slightly.

* Set all default settings to disabled by default for onboarding

* add extra logging attr. for debugging large groups of data.

* fix: listrr validation

* feat: rewrite of status page almost done

* add verbose logging in plex to debug looping

* revert debug logging for plex. figured out looping issue.

* add back plex log. add boilerplate for trakt content service. wip

* add back plex log. add boilerplate for trakt content service. wip

* feat: status page improvements (#169)

* Feat/better status page (#170)

* feat: status page improvements

* feat: status page improvements

* feat: status page improvements

* feat: new settings (#176)

* added more validation and logging

* changed name of test_items module

* refactor: edited minor things in settings (#177)

* remove tzdata

* refactor: edited minor things in settings (#177)

* remove tzdata

* fix typo on parser

* increase ratelimits on second_limiters

* Fix/parser/add attribute (#179)

* increase ratelimits on second_limiters

* Iceberg works. All scrapers working together. Symlinking works.

---------

Co-authored-by: Spoked <Spoked@localhost>

* add extra attrs to extended api endpoint

* feat: status page improvements (#182)

* [DEV] feat: frontend improvements (#193)

* feat: switched to new theme!
refactor: componentized most of form except tag inputs

* feat: completely refactored form and added many improvements in ui

* fix: fixed the mobile select issue

* feat: some more frontend improvements and bug fixes

* fix: shows not being downloaded

* fix: add check if data attr exists for orionoid

* fix: validate on empty symlink path correctly

* fix: added more validation to symlink paths

* feat: added some checks to save settings (#196)

* feat: improve symlink validation. fixed torrentio rate limits once and for all.

* fix: remove extra debug line from orionoid. correct settings path in symlink.

* fix: tweaked symlink validation.

* fix: raise jackett ratelimit

* fix: raise orionoid ratelimit

* feat: add back second limiter with 1/5s per on Torrentio. This will help prolong requests more instead of bursting.

* feat: frontend and backend improvements (#197) (#199)

* rework in progress.

* fix: correct limits for orionoid

* fix: switch to comprehensions

* fix: disable plex logging for id mismatches

* time for sleep. rework still wip.

* feat: parser works. needs more work. language needs a rewrite. disabled for now.

* fix: overseerr bug on using external ids

* fix: remove plex debug line for users

* disable tvdb checks from listrr, overseerr, plex. needs reworked.

* set torrentio to disabled by default. removed parse logs. raised torrentio limit slightly.

* Set all default settings to disabled by default for onboarding

* add extra logging attr. for debugging large groups of data.

* feat: started status page rewrite

* add dev branch builds with :dev tag (#165)

* fix: listrr validation

* rework in progress.

* fix: correct limits for orionoid

* fix: switch to comprehensions

* fix: disable plex logging for id mismatches

* time for sleep. rework still wip.

* feat: parser works. needs more work. language needs a rewrite. disabled for now.

* feat: frontend improvements (#158)

* feat: added global debug for settings

* feat: added dev to formDebug so it's always true in development but false in production

* feat: added DEBUG & LOG to general settings

* deps: switched svelte-sonner to shadcn customized toast component

* refactor: renamed PlexDebridItem to IcebergItem and added changes made to /items

* fix: fixed the wrong relative date in status page

* docs: readme improvements

* docs: readme improvements

* docs: readme improvements

* chore(deps): bump lucide-svelte from 0.303.0 to 0.307.0 in /frontend (#124)

* refactor: componentized forms, soon will do same for fields too

* Parse rewrite (#128)

* Move parser to its own module

* Add ORIGIN to env vars

* Fix overseerr, watchlist, jackett validation.

* Added more refined logic to parser module.

* Set stage for testing

* Add methods for individual checks

* Update sort logic

* Update default settings

* Fix jackett. Begin to add title support for jackett.

---------




* feat: onboarding on the way ;), major refactoring of form related code

* Simplified downloading logic and modified state matchine

* fix typo in state machine and handle movie pathing correctly

* Remove useless method

* Temporary fix to test

* Remove uncached stream hashes from item to avoid loop, some blacklisting logic could also be good

* chore(deps-dev): bump @typescript-eslint/eslint-plugin in /frontend (#134)

* chore(deps): bump lucide-svelte from 0.307.0 to 0.309.0 in /frontend (#133)

* chore(deps-dev): bump @sveltejs/kit from 2.0.1 to 2.3.2 in /frontend (#132)

* chore(deps): bump bits-ui from 0.13.0 to 0.14.0 in /frontend (#130)

* chore(deps-dev): bump @sveltejs/adapter-node in /frontend (#138)

* feat: some more onboarding and form improvements

* Dev startup to disabling pickling

* feat: Listrr Support Added (#136)

* Start Listrr Feature
* feat: Listrr ready for review.
* small tweaks. rewrite coming later.

---------



* Jackett rewrite (#139)

* Add TorBox scraper

* Add is_anime attribute to item

* Rework Jackett to Keyword Queries. Added categories. Removed Torbox

* Remove audio from parsing, it removed alot of good hits

* fix movie scraping and modify response parsing logic to be more readable

* fix: remove torbox module

* remove audio from being parsed

* remove more audio from parser

* fix typo

* fix: tidy audio and networks

* small tweaks

---------




* Avoid [None] if empty content service

* fix: handle bad quality manually in parser (#145)



* deps: updated deps due to security updateS

* feat: added more onboarding steps, some bugs also introduced

* chore(deps-dev): bump @sveltejs/kit from 2.3.2 to 2.4.2 in /frontend (#156)

* chore(deps-dev): bump prettier from 3.1.1 to 3.2.4 in /frontend (#155)

* chore(deps): bump lucide-svelte from 0.309.0 to 0.314.0 in /frontend (#154)

* chore(deps): bump bits-ui from 0.14.0 to 0.15.1 in /frontend (#153)

* feat: minor changes

* feat: deps change

* feat: deps change

* feat: onboarding MVP done

---------







* feat: frontend improvements (#159)

* feat: added global debug for settings

* feat: added dev to formDebug so it's always true in development but false in production

* feat: added DEBUG & LOG to general settings

* deps: switched svelte-sonner to shadcn customized toast component

* refactor: renamed PlexDebridItem to IcebergItem and added changes made to /items

* fix: fixed the wrong relative date in status page

* docs: readme improvements

* docs: readme improvements

* docs: readme improvements

* chore(deps): bump lucide-svelte from 0.303.0 to 0.307.0 in /frontend (#124)

* refactor: componentized forms, soon will do same for fields too

* Parse rewrite (#128)

* Move parser to its own module

* Add ORIGIN to env vars

* Fix overseerr, watchlist, jackett validation.

* Added more refined logic to parser module.

* Set stage for testing

* Add methods for individual checks

* Update sort logic

* Update default settings

* Fix jackett. Begin to add title support for jackett.

---------




* feat: onboarding on the way ;), major refactoring of form related code

* Simplified downloading logic and modified state matchine

* fix typo in state machine and handle movie pathing correctly

* Remove useless method

* Temporary fix to test

* Remove uncached stream hashes from item to avoid loop, some blacklisting logic could also be good

* chore(deps-dev): bump @typescript-eslint/eslint-plugin in /frontend (#134)

* chore(deps): bump lucide-svelte from 0.307.0 to 0.309.0 in /frontend (#133)

* chore(deps-dev): bump @sveltejs/kit from 2.0.1 to 2.3.2 in /frontend (#132)

* chore(deps): bump bits-ui from 0.13.0 to 0.14.0 in /frontend (#130)

* chore(deps-dev): bump @sveltejs/adapter-node in /frontend (#138)

* feat: some more onboarding and form improvements

* Dev startup to disabling pickling

* feat: Listrr Support Added (#136)

* Start Listrr Feature
* feat: Listrr ready for review.
* small tweaks. rewrite coming later.

---------



* Jackett rewrite (#139)

* Add TorBox scraper

* Add is_anime attribute to item

* Rework Jackett to Keyword Queries. Added categories. Removed Torbox

* Remove audio from parsing, it removed alot of good hits

* fix movie scraping and modify response parsing logic to be more readable

* fix: remove torbox module

* remove audio from being parsed

* remove more audio from parser

* fix typo

* fix: tidy audio and networks

* small tweaks

---------




* Avoid [None] if empty content service

* fix: handle bad quality manually in parser (#145)



* deps: updated deps due to security updateS

* feat: added more onboarding steps, some bugs also introduced

* chore(deps-dev): bump @sveltejs/kit from 2.3.2 to 2.4.2 in /frontend (#156)

* chore(deps-dev): bump prettier from 3.1.1 to 3.2.4 in /frontend (#155)

* chore(deps): bump lucide-svelte from 0.309.0 to 0.314.0 in /frontend (#154)

* chore(deps): bump bits-ui from 0.14.0 to 0.15.1 in /frontend (#153)

* feat: minor changes

* feat: deps change

* feat: deps change

* feat: onboarding MVP done

* refactor: moved schemes into forms/helpers.ts and command menu improvements

* refactor: switched to new font, changes made to all except status page

* refactor: minor change, didn't get commit

* fix: minor fix

* feat: fixed git merge conflicts issue

* feat: fixed status page font too

---------







* docs: minor improvements (#160)

* docs: readme improvements

* docs: readme improvements

* docs: minor improvements (#161)

* docs: readme improvements

* docs: readme improvements

* docs: readme improvements

* docs: minor improvements (#162)

* fix: correct parsing of external id's (#163)



* fix: overseerr bug on using external ids

* fix: remove plex debug line for users

* disable tvdb checks from listrr, overseerr, plex. needs reworked.

* set torrentio to disabled by default. removed parse logs. raised torrentio limit slightly.

* Set all default settings to disabled by default for onboarding

* add extra logging attr. for debugging large groups of data.

* fix: listrr validation

* feat: rewrite of status page almost done

* add verbose logging in plex to debug looping

* revert debug logging for plex. figured out looping issue.

* add back plex log. add boilerplate for trakt content service. wip

* add back plex log. add boilerplate for trakt content service. wip

* feat: status page improvements (#169)

* Feat/better status page (#170)

* feat: status page improvements

* feat: status page improvements

* feat: status page improvements

* feat: new settings (#176)

* added more validation and logging

* changed name of test_items module

* refactor: edited minor things in settings (#177)

* remove tzdata

* refactor: edited minor things in settings (#177)

* remove tzdata

* fix typo on parser

* increase ratelimits on second_limiters

* Fix/parser/add attribute (#179)

* increase ratelimits on second_limiters

* Iceberg works. All scrapers working together. Symlinking works.

---------



* add extra attrs to extended api endpoint

* feat: status page improvements (#182)

* [DEV] feat: frontend improvements (#193)

* feat: switched to new theme!
refactor: componentized most of form except tag inputs

* feat: completely refactored form and added many improvements in ui

* fix: fixed the mobile select issue

* feat: some more frontend improvements and bug fixes

* fix: shows not being downloaded

* fix: add check if data attr exists for orionoid

* fix: validate on empty symlink path correctly

* fix: added more validation to symlink paths

* feat: added some checks to save settings (#196)

* feat: improve symlink validation. fixed torrentio rate limits once and for all.

* fix: remove extra debug line from orionoid. correct settings path in symlink.

* fix: tweaked symlink validation.

* fix: raise jackett ratelimit

* fix: raise orionoid ratelimit

* feat: add back second limiter with 1/5s per on Torrentio. This will help prolong requests more instead of bursting.

---------

Co-authored-by: Spoked <Spoked@localhost>
Co-authored-by: KingPin <[email protected]>
Co-authored-by: Dreu LaVelle <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Spoked <[email protected]>
Co-authored-by: Dreu LaVelle <dreu.lavelle@localhost>

* fix: symlink works correctly now. missing arg in episode parsing.

* fix: remove container_path dir check

* chore: updated zod validations (#200)

* Rd_rewrite (#198)

* Update realdebrid.py

* fix: library path pointed to wrong path

* fix: better form validations and improvements (#202)

* fix: settings not saving in onboarding (#205)

* Use Pydantic for settings validation data model (#204)

* Use Pydantic data models for settings

* Rename Scraping to Scraper to be consistent across platform

* Revert "Rename Scraping to Scraper to be consistent across platform"

This reverts commit db29f42.

* fix data model

* Fix data model and references

* Add url field to torrentio model

* Correct docstring

* Refactor/validation (#211)

* fix: refactor run logic, tidy, validation and symlink improvements

* fix: symlink validation and paths. changed to users setting their own library dir.

* fix: update to latest settings

---------

Co-authored-by: Spoked <Spoked@localhost>

* fix: trakt_data not being generated. corrected empty new_items

* fix: simplify counter in logs

* fix: symlink pointed to wrong source path

* fix: lets make this a little cleaner for future reading..

* fix: couple tweaks

* refactor: content classes. wip.

* fix: media_items now properly is a container of your plex items

* fix: remove some unneccessary logging

* fix: remove redundant method for item removal. handles item removals now.

* fix: tidy removal excess logging

* remove: vscode settings. added library size logging to program class.

* revert: move plex back to init after Content

* fix: minor tweaks. add logging to scrapers. refactor is_cached method in rd.

* chore: apply black formatting

* push latest to dev. downloading only missing library items is fixed.

* small plex refactor to improve readability

* Add error log when no matching library paths are found in plex

* black formatting

* fix incorrect reference

* Rename and reformat

* final cleanup

* Improve scraping downloading concurrency (#224)

* cleanup

* validate refresh interval in the model

* use correct import and feild name

* incomplete not working code commit

* less broken now works to scraper stage

* Scrapers working

* End to end baby

* Settings reload

* Add update interval field for plex on frontend

* Cleanup and add type based item fetching

* make requested by based on class

* Render state name correctly

* add scheduler to reqs

* fill out any missing info in plex library using metadata state

* check for scheduler and executor when shutting down

* fix: plex validation

* dont instantiate container with list

* pickle doing bad things

* improve container updating accuraccy and add tests

* remove commented code

* state transition improvements

* don't check for metadata so often

* add indexed state and renamed content

* Use states instead of services to route items

* add type hints

* use hasattr instead

* Improvements from GB

* fix service reference

* state improvements

* feat: add symlink watcher using watchdog.

* improved deepcopy performance

* Infer parent_id when adding season/episode like direct parent link

* Lots of fixes

---------

Co-authored-by: Spoked <Spoked@localhost>

* only check if item has streams to be scraped state

* fix conditional that caused looping of symlinked state

* State machine testing (#236)

* improve state handling and warm boot

* finish method rename

* impove warm boot

* reorganize code to make it easier to run coverage

* make sure we also update item based on merged index state

* separate dev flag into cache and profile

* add failure condition to symlink

* add collection to get root item_id

* save state transition debug data

* add line number to executed_lines

---------

Co-authored-by: Spoked <Spoked@localhost>
Co-authored-by: Ayush Sehrawat <[email protected]>
Co-authored-by: KingPin <[email protected]>
Co-authored-by: Dreu LaVelle <[email protected]>
Co-authored-by: Ayush Sehrawat <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dreu LaVelle <dreu.lavelle@localhost>
Co-authored-by: Pukabyte <[email protected]>
Co-authored-by: Hank Bond <[email protected]>
  • Loading branch information
10 people authored Feb 27, 2024
1 parent b932df6 commit 86f6ad0
Show file tree
Hide file tree
Showing 61 changed files with 2,715 additions and 2,128 deletions.
1 change: 1 addition & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ COPY --from=frontend --chown=node:node /app/node_modules /iceberg/frontend/node_
COPY --from=frontend --chown=node:node /app/package.json /iceberg/frontend/package.json

# Backend
COPY VERSION /iceberg/VERSION
COPY backend/ /iceberg/backend
RUN python3 -m venv /venv
COPY requirements.txt /iceberg/requirements.txt
Expand Down
1 change: 1 addition & 0 deletions VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0.4.6
27 changes: 10 additions & 17 deletions backend/controllers/default.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from fastapi import APIRouter, Request
import requests
from utils.settings import settings_manager
from program.settings.manager import settings_manager


router = APIRouter(
Expand All @@ -13,6 +13,7 @@ async def root():
return {
"success": True,
"message": "Iceburg is running!",
"version": settings_manager.settings.version,
}


Expand All @@ -26,7 +27,7 @@ async def health(request: Request):

@router.get("/user")
async def get_rd_user():
api_key = settings_manager.get("real_debrid.api_key")
api_key = settings_manager.settings.real_debrid.api_key
headers = {"Authorization": f"Bearer {api_key}"}
response = requests.get(
"https://api.real-debrid.com/rest/1.0/user", headers=headers
Expand All @@ -37,19 +38,11 @@ async def get_rd_user():
@router.get("/services")
async def get_services(request: Request):
data = {}
if hasattr(request.app.program, "core_manager"):
for service in request.app.program.core_manager.services:
if hasattr(request.app.program, "services"):
for service in request.app.program.services.values():
data[service.key] = service.initialized
if getattr(service, "sm", False):
for sub_service in service.sm.services:
data[sub_service.key] = sub_service.initialized
if hasattr(request.app.program, "extras_manager"):
for service in request.app.program.extras_manager.services:
data[service.key] = service.initialized
if getattr(service, "sm", False):
for sub_service in service.sm.services:
data[sub_service.key] = sub_service.initialized
return {
"success": True,
"data": data
}
if not hasattr(service, "services"):
continue
for sub_service in service.services.values():
data[sub_service.key] = sub_service.initialized
return {"success": True, "data": data}
12 changes: 5 additions & 7 deletions backend/controllers/items.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from fastapi import APIRouter, HTTPException, Request
from program.media.state import MediaItemStates
from program.media.state import States

router = APIRouter(
prefix="/items",
Expand All @@ -12,15 +12,15 @@
async def get_states(request: Request):
return {
"success": True,
"states": [state for state in MediaItemStates],
"states": [state for state in States],
}


@router.get("/")
async def get_items(request: Request):
return {
"success": True,
"items": [item.to_dict() for item in request.app.program.media_items.items],
"items": [item.to_dict() for item in request.app.program.media_items],
}


Expand All @@ -38,6 +38,7 @@ async def get_extended_item_info(request: Request, item_id: str):
@router.delete("/remove/{item}")
async def remove_item(request: Request, item: str):
request.app.program.media_items.remove(item)
request.app.program.content.overseerr.delete_request(item)
return {
"success": True,
"message": f"Removed {item}",
Expand All @@ -49,7 +50,4 @@ async def get_imdb_info(request: Request, imdb_id: str):
item = request.app.program.media_items.get_item_by_imdb_id(imdb_id)
if item is None:
raise HTTPException(status_code=404, detail="Item not found")
return {
"success": True,
"item": item.to_extended_dict()
}
return {"success": True, "item": item.to_extended_dict()}
59 changes: 47 additions & 12 deletions backend/controllers/settings.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from copy import copy
from fastapi import APIRouter
from utils.settings import settings_manager
from fastapi import APIRouter, HTTPException
from program.settings.manager import settings_manager
from pydantic import BaseModel
from typing import Any, List

Expand Down Expand Up @@ -39,14 +39,25 @@ async def save_settings():
async def get_all_settings():
return {
"success": True,
"data": copy(settings_manager.get_all()),
"data": copy(settings_manager.settings),
}


@router.get("/get/{keys}")
async def get_settings(keys: str):
keys = keys.split(",")
data = {key: settings_manager.get(key) for key in keys}
@router.get("/get/{paths}")
async def get_settings(paths: str):
current_settings = settings_manager.settings.dict()
data = {}
for path in paths.split(","):
keys = path.split(".")
current_obj = current_settings

for k in keys:
if k not in current_obj:
return None
current_obj = current_obj[k]

data[path] = current_obj

return {
"success": True,
"data": data,
Expand All @@ -55,8 +66,32 @@ async def get_settings(keys: str):

@router.post("/set")
async def set_settings(settings: List[SetSettings]):
settings_manager.set(settings)
return {
"success": True,
"message": "Settings saved!",
}
current_settings = settings_manager.settings.dict()

for setting in settings:
keys = setting.key.split(".")
current_obj = current_settings

# Navigate to the last key's parent object, similar to the getter.
for k in keys[:-1]:
if k not in current_obj:
# If a key in the path does not exist, raise an exception or optionally create a new dict.
raise HTTPException(
status_code=400,
detail=f"Path '{'.'.join(keys[:-1])}' does not exist.",
)
current_obj = current_obj[k]

# Set the value at the final key.
if keys[-1] in current_obj:
current_obj[keys[-1]] = setting.value
else:
# If the final key does not exist, raise an exception.
raise HTTPException(
status_code=400,
detail=f"Key '{keys[-1]}' does not exist in path '{'.'.join(keys[:-1])}'.",
)

settings_manager.load(settings_dict=current_settings)

return {"success": True, "message": "Settings updated successfully."}
32 changes: 24 additions & 8 deletions backend/main.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,32 @@
import contextlib
import sys
import threading
import time
import argparse
import traceback

import uvicorn
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from program import Program
from controllers.settings import router as settings_router
from controllers.items import router as items_router
from controllers.default import router as default_router
import contextlib
import sys
import threading
import time
import uvicorn
import argparse

from utils.logger import logger

parser = argparse.ArgumentParser()
parser.add_argument('--dev', action='store_true', help='Enable development mode')
parser.add_argument(
"--ignore_cache", action="store_true",
help="Ignore the cached metadata, create new data from scratch."
)
parser.add_argument(
"--profile_state_transitions", action="store_true",
help="Use a profiling process to determine what paths the state machine took"
)
args = parser.parse_args()


class Server(uvicorn.Server):
def install_signal_handlers(self):
pass
Expand All @@ -27,11 +39,15 @@ def run_in_thread(self):
while not self.started:
time.sleep(1e-3)
yield
except Exception as e:
logger.error(traceback.format_exc())
raise e
finally:
app.program.stop()
self.should_exit = True
sys.exit(0)


app = FastAPI()
app.program = Program(args)

Expand All @@ -55,4 +71,4 @@ def run_in_thread(self):
app.program.start()
app.program.run()
except KeyboardInterrupt:
pass
pass
64 changes: 1 addition & 63 deletions backend/program/__init__.py
Original file line number Diff line number Diff line change
@@ -1,66 +1,4 @@
"""Program main module"""
import os
import threading
import time
import concurrent.futures
from program.scrapers import Scraping
from program.realdebrid import Debrid
from program.symlink import Symlinker
from program.media.container import MediaItemContainer
from utils.logger import logger, get_data_path
from program.plex import Plex
from program.content import Content
from utils.utils import Pickly
from utils.settings import settings_manager as settings
from utils.service_manager import ServiceManager
from program.program import Program, Event


class Program(threading.Thread):
"""Program class"""

def __init__(self, args):
super().__init__(name="Iceberg")
self.running = False
self.startup_args = args

def start(self):
logger.info("Iceberg v%s starting!", settings.get("version"))
self.initialized = False
self.media_items = MediaItemContainer(items=[])
self.data_path = get_data_path()
if not os.path.exists(self.data_path):
os.mkdir(self.data_path)
if not self.startup_args.dev:
self.pickly = Pickly(self.media_items, self.data_path)
self.pickly.start()
self.core_manager = ServiceManager(self.media_items, True, Content, Plex, Scraping, Debrid, Symlinker)
if self.validate():
logger.info("Iceberg started!")
else:
logger.info("----------------------------------------------")
logger.info("Iceberg is waiting for configuration to start!")
logger.info("----------------------------------------------")
super().start()
self.running = True
self.initialized = True

def run(self):
while self.running:
if self.validate():
with concurrent.futures.ThreadPoolExecutor(
max_workers=10, thread_name_prefix="Worker"
) as executor:
for item in self.media_items:
executor.submit(item.perform_action, self.core_manager.services)
time.sleep(1)

def validate(self):
return all(service.initialized for service in self.core_manager.services)

def stop(self):
for service in self.core_manager.services:
if getattr(service, "running", False):
service.stop()
self.pickly.stop()
settings.save()
self.running = False
40 changes: 1 addition & 39 deletions backend/program/content/__init__.py
Original file line number Diff line number Diff line change
@@ -1,42 +1,4 @@
import threading
import time
from utils.logger import logger
from utils.service_manager import ServiceManager
from .mdblist import Mdblist
from .overseerr import Overseerr
from .plex_watchlist import PlexWatchlist
from .listrr import Listrr


class Content(threading.Thread):
def __init__(self, media_items):
super().__init__(name="Content")
self.initialized = False
self.key = "content"
self.running = False
self.sm = ServiceManager(media_items, False, Overseerr, PlexWatchlist, Listrr, Mdblist)
if not self.validate():
logger.error("You have no content services enabled, please enable at least one!")
return
self._get_content()
self.initialized = True

def validate(self):
return any(service.initialized for service in self.sm.services)

def run(self) -> None:
while self.running:
self._get_content()
time.sleep(1)

def _get_content(self) -> None:
for service in self.sm.services:
if service.initialized:
service.run()

def start(self) -> None:
self.running = True
super().start()

def stop(self) -> None:
self.running = False
from .listrr import Listrr
Loading

0 comments on commit 86f6ad0

Please sign in to comment.