Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for MultiField on GPU #201

Open
wants to merge 71 commits into
base: develop
Choose a base branch
from
Open

Add support for MultiField on GPU #201

wants to merge 71 commits into from

Commits on Apr 29, 2024

  1. Add RegisterPointerInfo

    @wdeconinck
    wdeconinck committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    a12114f View commit details
    Browse the repository at this point in the history

  2. 1) sync_host_device to skip unused fields; 2) xx-needs-update means y… 

    …y-does-not-need-update
    @sbrdar @wdeconinck
    sbrdar authored and wdeconinck committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    22d7fb7 View commit details
    Browse the repository at this point in the history

  3. add FielSet cummulative gpu operations: allocate_device, update_devic… 

    …e ...
    @sbrdar @wdeconinck
    sbrdar authored and wdeconinck committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    b7c37d1 View commit details
    Browse the repository at this point in the history

  4. Fix atlas_fctest_field_gpu_device with gt storage

    @wdeconinck
    wdeconinck committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    3b582bc View commit details
    Browse the repository at this point in the history

  5. Use standard CMake find_package(OpenACC) instead of handcrafted flags

    @wdeconinck
    wdeconinck committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    e59fe9c View commit details
    Browse the repository at this point in the history

  6. Trace GPU events when ATLAS_TRACE_MEMORY=1

    @wdeconinck
    wdeconinck committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    d74086e View commit details
    Browse the repository at this point in the history

  7. 1) Pass a config argument from Field down to Array (defaulted to util… 

    …::Config()); 2) Implement the optional host memory pinning and the optional mapping of this pinned memory for faster device memory access
    @sbrdar @wdeconinck
    sbrdar authored and wdeconinck committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    e41689b View commit details
    Browse the repository at this point in the history

  8. Disable atlas_test_field_pinning, as test_field_pinning.cc not found

    @wdeconinck
    wdeconinck committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    590d37d View commit details
    Browse the repository at this point in the history

  9. Add copy of Config for Fortran

    @sbrdar
    sbrdar committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    c2162f7 View commit details
    Browse the repository at this point in the history

  10. forgot to add unit test for pinning memory of Field

    @sbrdar
    sbrdar committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    2f8f0d9 View commit details
    Browse the repository at this point in the history

  11. add -options- argument in functionspace%create_field; adapt unit test… 

    … fctest_functionspace
    @sbrdar
    sbrdar committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    3003ce6 View commit details
    Browse the repository at this point in the history

  12. also add options in create_field called from atlas_functionspace_Bloc… 

    …kStructureColumns
    @sbrdar
    sbrdar committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    d6ac142 View commit details
    Browse the repository at this point in the history

  13. trace report for Field with memory pinning; 2) remove pinnen memory r… 

    …eport from FieldImpl - it is an Array thing
    @sbrdar
    sbrdar committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    0548225 View commit details
    Browse the repository at this point in the history

  14. enforced pinning via eckit::Resource (by Willem)

    @sbrdar
    sbrdar committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    8fcecba View commit details
    Browse the repository at this point in the history

  15. make the options argument pass down to ArrayDataStore

    @sbrdar
    sbrdar committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    5f4b5f9 View commit details
    Browse the repository at this point in the history

  16. move host_memory booleans to ArrayDataStore

    @sbrdar
    sbrdar committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    946fef6 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. Tidy, revert host_memory_pinning to NativeDataStore

    @wdeconinck
    wdeconinck committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    1576efc View commit details
    Browse the repository at this point in the history

  2. Fix test

    @wdeconinck
    wdeconinck committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    9f64a4f View commit details
    Browse the repository at this point in the history

Commits on May 8, 2024

  1. add thorough tests on memory pinning and mapping of Field

    @sbrdar
    sbrdar committed May 8, 2024
    Configuration menu
    Copy the full SHA
    5c74b9c View commit details
    Browse the repository at this point in the history

  2. a working c++ usage of field->device_data in ACC-deviceptr (not nice … 

    …though)
    @sbrdar
    sbrdar committed May 8, 2024
    Configuration menu
    Copy the full SHA
    a6ba9c3 View commit details
    Browse the repository at this point in the history

  3. 1) add field%device_data and Fortran API backend; 2) improve the unit… 

    … test to deal with acc-present and acc-deviceptr
    @sbrdar
    sbrdar committed May 8, 2024
    Configuration menu
    Copy the full SHA
    76fa5ff View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. fix the unit test - remove external_acc_routine with device pointer a… 

    …rgument in Fortran
    @sbrdar
    sbrdar committed May 13, 2024
    Configuration menu
    Copy the full SHA
    49ab5b8 View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. bug fix in calling for device_data from Field_module.fypp (thx Willem)

    @sbrdar
    sbrdar committed May 15, 2024
    Configuration menu
    Copy the full SHA
    cc2ad95 View commit details
    Browse the repository at this point in the history

  2. improve the Fortran unit test for wrapping of discontiguous data

    @sbrdar
    sbrdar committed May 15, 2024
    Configuration menu
    Copy the full SHA
    ff2cb5f View commit details
    Browse the repository at this point in the history

  3. strides on the device do not correspond to the device shape in case o… 

    …f discontiguous data, so that device_stridesf had to be introduced
    @sbrdar
    sbrdar committed May 15, 2024
    Configuration menu
    Copy the full SHA
    20220b0 View commit details
    Browse the repository at this point in the history

  4. using cudaMemcpy2D to do strided copy of discontiguous data

    @sbrdar
    sbrdar committed May 15, 2024
    Configuration menu
    Copy the full SHA
    f19e676 View commit details
    Browse the repository at this point in the history

  5. add c-test for wrapping discontiguous data (with Willem)

    @sbrdar
    sbrdar committed May 15, 2024
    Configuration menu
    Copy the full SHA
    4d8ed39 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2024

  1. update unit test for wrapping of the discontiguous data; fix test_field

    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    cb13d32 View commit details
    Browse the repository at this point in the history

  2. bug fix in wrapping of discontiguous data; make unit test on wrapping… 

    … pass
    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    744e68f View commit details
    Browse the repository at this point in the history

  3. bug fix in wrapping discontiguous data to make Fortran and C++ unit t… 

    …est pass
    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    8d38b9d View commit details
    Browse the repository at this point in the history

  4. add all possible tests for slicing of a 4d array for the wrapping of … 

    …a discontiguous data on GPU; remove debug output
    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    c6142b7 View commit details
    Browse the repository at this point in the history

  5. Introducing MultiField

    @wdeconinck @sbrdar
    wdeconinck authored and sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    cf72be7 View commit details
    Browse the repository at this point in the history

  6. Support for vector fields in multifield

    @wdeconinck @sbrdar
    wdeconinck authored and sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    5929613 View commit details
    Browse the repository at this point in the history

  7. Use util::Factory for MultiFieldCreatorFactory

    @wdeconinck @sbrdar
    wdeconinck authored and sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    0a9e45a View commit details
    Browse the repository at this point in the history

  8. MultiField with ArrayAllocator

    @wdeconinck @sbrdar
    wdeconinck authored and sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    5762a54 View commit details
    Browse the repository at this point in the history

  9. provide nproma in addition to block_size, since the first block_size … 

    …is not necessarally equal nproma
    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    1884eda View commit details
    Browse the repository at this point in the history

  10. add Fortran interface and a unit test for multifield

    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    be750f9 View commit details
    Browse the repository at this point in the history

  11. Add FieldObserver::onFieldDestruction

    @wdeconinck @sbrdar
    wdeconinck authored and sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    c745473 View commit details
    Browse the repository at this point in the history

  12. Use MultiFieldArrayRegistry to avoid accidental deletion of multifiel… 

    …d arrays
    @wdeconinck @sbrdar
    wdeconinck authored and sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    102c01d View commit details
    Browse the repository at this point in the history

  13. fix namespace

    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    721672a View commit details
    Browse the repository at this point in the history

  14. extend the Fortran interface of MultiField to return its FieldSet; ex… 

    …tend FieldSet to concatenate two FieldSets
    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    ff04675 View commit details
    Browse the repository at this point in the history

  15. a unit test for FieldSet concatenation

    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    86a961c View commit details
    Browse the repository at this point in the history

  16. extend the unit tests for FieldSet

    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    f27eb06 View commit details
    Browse the repository at this point in the history

  17. 1) turn fypp to F90; 2) add a Fortran unit test for MultiField

    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    8c233c9 View commit details
    Browse the repository at this point in the history

  18. remove file

    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    b6087cf View commit details
    Browse the repository at this point in the history

  19. improve Fortran unit test for MultiField

    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    798c4bd View commit details
    Browse the repository at this point in the history

  20. add data access by array and index for MultiField plus a unit test fo… 

    …r it
    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    1e15af0 View commit details
    Browse the repository at this point in the history

  21. make compile again

    @sbrdar
    sbrdar committed May 17, 2024
    Configuration menu
    Copy the full SHA
    0ab9946 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. add device_data on FieldSet in Fortran API

    @sbrdar
    sbrdar committed May 21, 2024
    Configuration menu
    Copy the full SHA
    c35c13b View commit details
    Browse the repository at this point in the history

  2. bug fix, do not attempt the acc_map when da^C discontiguous

    @sbrdar
    sbrdar committed May 21, 2024
    Configuration menu
    Copy the full SHA
    5f16f02 View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. pass config through MultiField down to Array (with Willem)

    @sbrdar
    sbrdar committed May 22, 2024
    Configuration menu
    Copy the full SHA
    96e2918 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'feature/gpu_discontiguous_data' of github.com:ecmwf/atl… 

    …as into feature/gpu_discontiguous_data
    @sbrdar
    sbrdar committed May 22, 2024
    Configuration menu
    Copy the full SHA
    baa4cd3 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. WrappedArray has to know if the device memory is mapped to the host m… 

    …emory (with Willem)
    @sbrdar
    sbrdar committed May 23, 2024
    Configuration menu
    Copy the full SHA
    9570229 View commit details
    Browse the repository at this point in the history

  2. make compile on GH200 (by Willem)

    @sbrdar
    sbrdar committed May 23, 2024
    Configuration menu
    Copy the full SHA
    8db9b44 View commit details
    Browse the repository at this point in the history

  3. make pinning work correctly; add tracing to Array cuda calls (with Wi… 

    …llem)
    @sbrdar
    sbrdar committed May 23, 2024
    Configuration menu
    Copy the full SHA
    33083ef View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. restructure MultiField construction (with Willem)

    @sbrdar
    sbrdar committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    179e07d View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. fix MultiField constructor for C based only on datatype, shape, var_n… 

    …ames
    @sbrdar
    sbrdar committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    f55c748 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. add MultiFieldCreatorArray - a generic MultiField constructor from da… 

    …tatype, shape, variable_names
    @sbrdar
    sbrdar committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    5d30f10 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. MultiFieldCreatorIFS does not need this constructor

    @sbrdar
    sbrdar committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    715125a View commit details
    Browse the repository at this point in the history

  2. Config-constructor for MultiFieldCreatorArray

    @sbrdar
    sbrdar committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    3d9b6e9 View commit details
    Browse the repository at this point in the history

  3. add problematic multifield-array test with config

    @sbrdar
    sbrdar committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    6fe2801 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'develop' into feature/gpu_discontiguous_data

    @sbrdar
    sbrdar committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    e9546be View commit details
    Browse the repository at this point in the history

  5. Merge branch 'develop' into feature/gpu_discontiguous_data

    @sbrdar
    sbrdar committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    f216567 View commit details
    Browse the repository at this point in the history

  6. remove config argument in many Field and Array constructors

    @sbrdar
    sbrdar committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    4f2092c View commit details
    Browse the repository at this point in the history

  7. bug fix in the IFS field creator

    @sbrdar
    sbrdar committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    b84a58a View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. bypass Intel compiler problem

    @sbrdar
    sbrdar committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    7f33861 View commit details
    Browse the repository at this point in the history

  2. bug fix in MultiFieldCreatorArray

    @sbrdar
    sbrdar committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    3153f92 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. fix MultiField Fortran API constructor to fall back to C-implementation

    @sbrdar
    sbrdar committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    b666b4d View commit details
    Browse the repository at this point in the history

  2. remove debug output

    @sbrdar
    sbrdar committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    5f16559 View commit details
    Browse the repository at this point in the history

  3. remove debug output

    @sbrdar
    sbrdar committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    946ed54 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'develop' into feature/gpu_discontiguous_data

    @sbrdar
    sbrdar committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    e84dc2e View commit details
    Browse the repository at this point in the history