'file_size' KeyError at /api/datasets/

[liviust]

There seems to be an error with the datasets 'file_size' Key, both in develop and master branches. It triggers when uploading a bundle competition.

2024-06-09 09:58:59 django-1          | Internal Server Error: /api/datasets/
2024-06-09 09:58:59 django-1          | Traceback (most recent call last):
2024-06-09 09:58:59 django-1          |   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272506+00:00 [info] <0.254.0> Running boot step rabbit_exchange_type_headers defined by app rabbit
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272605+00:00 [info] <0.254.0> Running boot step rabbit_exchange_type_topic defined by app rabbit
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272658+00:00 [info] <0.254.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit
2024-06-09 09:58:59 django-1          |     response = get_response(request)
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272684+00:00 [info] <0.254.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit
2024-06-09 09:58:59 django-1          |   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
2024-06-09 09:58:59 django-1          |     response = self.process_exception_by_middleware(e, request)
2024-06-09 09:58:59 django-1          |   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
2024-06-09 09:58:59 django-1          |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272700+00:00 [info] <0.254.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit
2024-06-09 09:58:59 django-1          |   File "/usr/local/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
2024-06-09 09:58:59 django-1          |     return view_func(*args, **kwargs)
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272771+00:00 [info] <0.254.0> Running boot step rabbit_priority_queue defined by app rabbit
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272803+00:00 [info] <0.254.0> Priority queues enabled, real BQ is rabbit_variable_queue
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272844+00:00 [info] <0.254.0> Running boot step rabbit_queue_location_client_local defined by app rabbit
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272875+00:00 [info] <0.254.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272891+00:00 [info] <0.254.0> Running boot step rabbit_queue_location_random defined by app rabbit
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272906+00:00 [info] <0.254.0> Running boot step kernel_ready defined by app rabbit
2024-06-09 09:58:59 django-1          |   File "/usr/local/lib/python3.8/site-packages/rest_framework/viewsets.py", line 116, in view
2024-06-09 09:58:59 django-1          |     return self.dispatch(request, *args, **kwargs)
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272913+00:00 [info] <0.254.0> Running boot step rabbit_sysmon_minder defined by app rabbit
2024-06-09 09:58:59 django-1          |   File "/usr/local/lib/python3.8/site-packages/rest_framework/views.py", line 495, in dispatch
2024-06-09 09:58:59 django-1          |     response = self.handle_exception(exc)
2024-06-09 09:53:40 rabbit-1          | 2024-06-09 06:53:40.272968+00:00 [info] <0.254.0> Running boot step rabbit_epmd_monitor defined by app rabbit
2024-06-09 09:58:59 django-1          |   File "/usr/local/lib/python3.8/site-packages/rest_framework/views.py", line 455, in handle_exception
2024-06-09 09:58:59 django-1          |     self.raise_uncaught_exception(exc)
2024-06-09 09:58:59 django-1          |   File "/usr/local/lib/python3.8/site-packages/rest_framework/views.py", line 492, in dispatch
2024-06-09 09:58:59 django-1          |     response = handler(request, *args, **kwargs)
2024-06-09 09:58:59 django-1          |   File "/app/src/apps/api/views/datasets.py", line 85, in create
2024-06-09 09:58:59 django-1          |     file_size = float(request.data['file_size'])

[liviust]

Interestingly enough, the error goes away if you first upload a submission via the Resources menu and then create a competition. Therefore, the above error happens when you compile the two branches from scratch and upload a competition as the first command.

After you upload a submission, the error goes away, and you can upload a competition bundle. However, when uploading a submission to the newly created competition, you'll get:

Rriot+compiler.min.js:2  Cannot read properties of undefined (reading 'length')
riot+compiler.min.js:2 <comp-tabs> {competition.files.length != 0}

And then:

Object
competition: 
admin: true
admin_privilege: true
allow_robot_submissions: false
auto_run_submissions: true
can_participants_make_submissions_public: true
collaborators: []
competition_type: "competition"
contact_email: ""
created_by: "admin"
created_when: "2024-06-09T07:15:12.906106Z"
description: "The well known Iris dataset from Fisher's classic paper (Fisher, 1936)."
docker_image: "codalab/codalab-legacy:py37"
enable_detailed_results: true
fact_sheet: null
files: Array(8)
0: {key: '2a39495b-8859-470a-bc03-24c20fbd00fb', name: "ingestion_program @ '06-09-2024 07:15'", file_size: '28.86', phase: 'Development', task: 'Development Task', …}
1: {key: '348a386e-e4c0-43e6-bf0f-d3f60e93395c', name: "scoring_program @ '06-09-2024 07:15'", file_size: '19.68', phase: 'Development', task: 'Development Task', …}
2: {key: '29e28217-f907-4a38-9427-e0e62c03b415', name: "input_data @ '06-09-2024 07:15'", file_size: '1.84', phase: 'Development', task: 'Development Task', …}
3: {key: '254f93b1-475f-4728-bc7e-e76013eaf806', name: "reference_data @ '06-09-2024 07:15'", file_size: '0.16', phase: 'Development', task: 'Development Task', …}
4: {key: '7275821a-cd96-475b-b03b-0c9fd8ac60d3', name: "ingestion_program @ '06-09-2024 07:15'", file_size: '28.86', phase: 'Final', task: 'Final Task', …}
5: {key: '77f1ba0b-5075-482f-801c-c29bbba1a3d5', name: "scoring_program @ '06-09-2024 07:15'", file_size: '19.68', phase: 'Final', task: 'Final Task', …}
6: {key: 'd9458b50-3261-4c3c-bdaa-806e1690c6e8', name: "input_data @ '06-09-2024 07:15'", file_size: '1.84', phase: 'Final', task: 'Final Task', …}
7: {key: 'b40e7890-71bb-41a6-9c0e-30bc02a33845', name: "reference_data @ '06-09-2024 07:15'", file_size: '0.17', phase: 'Final', task: 'Final Task', …}
length: 8
[[Prototype]]: Array(0)
...

[liviust]

Also, the secret_key and few other properties should not be accessible to unauthorized users. I was surprised to be able to extract it using the browser console, as it is printed there. You can do this for any competition, even if you are not logged in. If an organizer decides to unpublish a competition, the users can still access it via the secret key which cancels the logic to be able to unpublish it.

Another concerning info that is leaked is the whitelist_emails.

[Didayolo]

Maybe related to:

• Changing file_size value in Django admin interface is not working. #1468

Another concerning info that is leaked is the whitelist_emails

Indeed:

• White list emails leaked #1366

[ihsaan-ullah]

I cannot reproduce the issue you have mentioned about file_size. Here is what I did:

1. Using latest develop branch
2. Run docker-compose up -d
3. Run docker-compose exec django ./manage.py migrate
4. Run docker-compose exec django ./manage.py collectstatic --noinput
5. Access codabench using localhost
6. create a new user using Signup
7. Activate this user using the activation link from django container
8. Login
9. Go to Benchmarks -> Management, upload this bundle: iris_competition_bundle.zip

I see no error in django container

@liviust if you can reproduce the issue, please share the steps here.

[ihsaan-ullah]

Todos:

• Fix competition printing in console
  Solved by: Competition console prints removed #1539
• Fix file_size error