About the readability

[familyld]

Hi Aviral,

In the paper, you claim CQL can be implemented with less than 20 lines of code, but it's really difficult to identify these "20 lines of code" from the current version of your project which is built upon other projects. Would you please point out which part of code exactly corresponds to the core of CQL? I really like your idea of CQL, both the theoretical part and its simplicity, but currently, it seems very hard to follow.

Best,
Zhi-Hong

[familyld]

For the Atari experiments, it seems like line 221 ~ 243 in quantile_agent.py are of interest.

For the D4RL experiments, it seems like line 233 ~ 300 in cql.py are of interest, but more lines of code are modified or added, comparing to sac.py.

Since the two experiments are based on different methods (QR-DQN and SAC) and different libraries (dopamine and rlkit), the detailed implementations are quite different. It would be great if the author can add more comments and explicitly correspond the code to the equations in the paper, e..g., Equation (4).

[aviralkumar2907]

Hello @familyld,

Yes, I hope to add some more comments in about a month timeline. Sorry for the delay in doing that.