Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(off-policy): fix log when not update #206

Merged
merged 77 commits into from
Apr 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
4987e80
refactor: change architecture of omnisafe (#121)
rockmagma02 Feb 23, 2023
90f7549
fix(algo): fix no return in algo_wrapper::learn (#122)
rockmagma02 Feb 27, 2023
16be6a8
chore: support num_thread setting (#124)
Gaiejj Feb 28, 2023
51e7f96
feat: update architecture of config.yaml (#126)
zmsn-2077 Mar 5, 2023
2aa1417
fix(logger, wrapper): support csv file and velocity tasks (#131)
Gaiejj Mar 6, 2023
04477eb
chore: fix typo. (#134)
1Asan Mar 6, 2023
fe686bb
fix(ppo): fix entropy loss (#135)
Gaiejj Mar 7, 2023
63dc035
test: add more test case, and fix bugs (#136)
rockmagma02 Mar 9, 2023
8b32d73
fix: support new config for exp_grid (#142)
muchvo Mar 9, 2023
06dcbdb
feat: support policy evaluation (#137)
Gaiejj Mar 10, 2023
c232fcf
feat(off-policy): add DDPG, TD3 SAC (#128)
Gaiejj Mar 13, 2023
78ec6bd
fix(rollout, exp_grid): fix logdir path conflict (#145)
muchvo Mar 13, 2023
817edf9
fix(on-policy): fix the second order algorithms performance (#147)
Gaiejj Mar 13, 2023
d7233ce
chore(on-policy): update benchmark performance for first-order algori…
muchvo Mar 14, 2023
cb633d6
docs: update README.md (#149)
Gaiejj Mar 14, 2023
ba48512
docs: update changelog and release v0.1.0
zmsn-2077 Mar 15, 2023
3e8de48
feat: support off-policy lagrangian algorithms
Gaiejj Mar 16, 2023
8463ca5
chore: update yaml (#155)
Gaiejj Mar 17, 2023
5068262
chore: update github workflow (#156)
Gaiejj Mar 18, 2023
6c1ed1d
feat: support cuda
Gaiejj Mar 18, 2023
3c2daf0
feat: support command line interfaces for omnisafe (#144)
muchvo Mar 18, 2023
3380d75
feat: support cuda exp grid
Gaiejj Mar 19, 2023
f43f32c
chore: merge dev branch
Gaiejj Mar 19, 2023
973b363
fix(onpolicy_adapter): fix the calculation of last state value (#162)
Dtrc2207 Mar 19, 2023
cb1544c
Merge branch 'dev' into dev-action
Gaiejj Mar 19, 2023
2a294a4
fix: [pre-commit.ci] auto fixes [...]
pre-commit-ci[bot] Mar 19, 2023
065e22d
fix: disable test cuda
Gaiejj Mar 19, 2023
60c901e
chore: update yaml (#155)
Gaiejj Mar 17, 2023
d955dd8
chore: update github workflow (#156)
Gaiejj Mar 18, 2023
12dbced
feat: support command line interfaces for omnisafe (#144)
muchvo Mar 18, 2023
eb45923
fix(onpolicy_adapter): fix the calculation of last state value (#162)
Dtrc2207 Mar 19, 2023
335f424
Merge branch 'PKU-MARL:dev' into dev
Gaiejj Mar 20, 2023
e722980
chore: rebase dev branch
Gaiejj Mar 20, 2023
d63c4f2
Merge branch 'dev-action' of https://github.com/Gaiejj/omnisafe_zjy i…
Gaiejj Mar 20, 2023
4d96546
fix(onpolicy_adapter): fix the calculation of last state value (#164)
muchvo Mar 20, 2023
2ba25ff
feat: support cuda
Gaiejj Mar 18, 2023
a2cbdf5
feat: support cuda exp grid
Gaiejj Mar 19, 2023
1debdf1
fix: [pre-commit.ci] auto fixes [...]
pre-commit-ci[bot] Mar 19, 2023
1dc9a9d
fix: disable test cuda
Gaiejj Mar 19, 2023
708496d
Merge branch 'dev' of https://github.com/Gaiejj/omnisafe_zjy into dev
Gaiejj Mar 20, 2023
9687866
Merge branch 'dev' into dev-action
Gaiejj Mar 20, 2023
7e68e01
Merge branch 'dev-action' of https://github.com/Gaiejj/omnisafe_zjy i…
Gaiejj Mar 20, 2023
0a647c2
chore: update yaml
Gaiejj Mar 20, 2023
9a2887a
Merge branch 'dev-action' into dev-backup
Gaiejj Mar 20, 2023
6ea6758
docs: update README.md
Gaiejj Mar 20, 2023
2f04f4a
feat: support cuda (#163)
Gaiejj Mar 20, 2023
9fa17e5
docs: update docs style (#169)
Gaiejj Mar 24, 2023
5e66a6e
fix(config.py): fix config assertion (#174)
Gaiejj Mar 24, 2023
9ffdb0e
chore: fix typo in readme (#172)
rockmagma02 Mar 24, 2023
45a3a49
chore: merge dev branch
Gaiejj Mar 24, 2023
f634615
feat: support cvpo algorithm
Gaiejj Mar 24, 2023
c730cba
chore: merge dev branch
Gaiejj Mar 24, 2023
84816a6
chore: update yaml
Gaiejj Mar 24, 2023
eff0672
fix: fix lag methods
Gaiejj Mar 26, 2023
8cf2687
feat: add `ruff` and `codespell` integration (#186)
XuehaiPan Mar 28, 2023
989e11e
docs: update changelog.md (#187)
zmsn-2077 Mar 29, 2023
1574db6
deps(torch): pin the version of pytorch (#184)
muchvo Mar 29, 2023
a420d39
chore: merge dev
Gaiejj Mar 30, 2023
07b4aac
fix: fix off-policy lag
Gaiejj Mar 30, 2023
14f5493
feat: perfecting training workflow (#185)
muchvo Mar 30, 2023
e121a8b
fix: fix off_policy algorithm
Gaiejj Mar 30, 2023
03103be
fix: fix cvpo
Gaiejj Apr 1, 2023
c6b9e2e
refactor: refactor off policy lag
Gaiejj Apr 5, 2023
3d6e7c5
chore: merge dev
Gaiejj Apr 5, 2023
2b34c94
chore: update configs
Gaiejj Apr 5, 2023
26fa1c7
chore: rebase dev
Gaiejj Apr 5, 2023
5bd5e21
chore: clean the code
Gaiejj Apr 5, 2023
d05f0ce
chore: clean the code
Gaiejj Apr 5, 2023
2a7d7a8
chore: update yaml
Gaiejj Apr 5, 2023
b800909
chore: clean the code
Gaiejj Apr 5, 2023
8c5a8f2
chore: update env version
Gaiejj Apr 5, 2023
c60aedf
chore: clean the code
Gaiejj Apr 5, 2023
f31a3e1
chore: merge dev
Gaiejj Apr 5, 2023
6e71cee
fix: fix device error
Gaiejj Apr 6, 2023
746371f
docs(adapters): update comment
Gaiejj Apr 6, 2023
e4a9d0a
fix(off_policy): fix log when not update
Gaiejj Apr 6, 2023
bc10dc5
chroe: merge dev
Gaiejj Apr 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions omnisafe/algorithms/off_policy/sac_lag.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,6 @@ def _log_when_not_update(self) -> None:
super()._log_when_not_update()
self._logger.store(
**{
'Loss/Loss_cost_critic': 0.0,
'Value/cost_critic': 0.0,
'Metrics/LagrangeMultiplier': self._lagrange.lagrangian_multiplier.item(),
'Metrics/LagrangeMultiplier': self._lagrange.lagrangian_multiplier.data.item(),
},
)
8 changes: 8 additions & 0 deletions omnisafe/algorithms/off_policy/td3_lag.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,11 @@ def _loss_pi(
loss_c = self._lagrange.lagrangian_multiplier.item() * loss_q_c

return (loss_r + loss_c).mean() / (1 + self._lagrange.lagrangian_multiplier.item())

def _log_when_not_update(self) -> None:
super()._log_when_not_update()
self._logger.store(
**{
'Metrics/LagrangeMultiplier': self._lagrange.lagrangian_multiplier.data.item(),
},
)