Skip to content

Commit

Permalink
Merge branch 'yeshwanth/component_release_with_env75' of https://gith…
Browse files Browse the repository at this point in the history
…ub.com/Azure/azureml-assets into yeshwanth/component_release_with_env75
  • Loading branch information
yeshsurya committed Oct 21, 2024
2 parents d83872e + 46516eb commit c71ab53
Show file tree
Hide file tree
Showing 8 changed files with 25 additions and 13 deletions.
9 changes: 8 additions & 1 deletion assets/models/system/cxrreportgen/description.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

The CXRReportGen model utilizes a multimodal architecture, integrating a BiomedCLIP image encoder with a Phi-3-Mini text encoder to help an application interpret complex medical imaging studies of chest X-rays. CXRReportGen follows the same framework as **[MAIRA-2](https://www.microsoft.com/en-us/research/publication/maira-2-grounded-radiology-report-generation/)**. When built upon and integrated into an application, CXRReportGen may help developers generate comprehensive and structured radiology reports, with visual grounding represented by bounding boxes on the images.

This repository contains the CXRReportGen model, which is packaged in MLflow format and deployed using Azure ML service. The estimated time to package and begin to build upon the model is approximately 1 hour.

This model is intended and provided as-is for research and model development exploration. CXRReportGen is not designed or intended to be deployed in clinical settings as-is nor is it intended for use in the diagnosis or treatment of any health or medical condition (including generating radiology reports for use in patient care), and the model’s performance for such purposes has not been established.
You bear sole responsibility and liability for any use of CXRReportGen, including verification of outputs and incorporation into any product or service intended for a medical purpose or to inform clinical decision-making, compliance with applicable healthcare laws and regulations, and obtaining any necessary clearances or approvals.

For documentation and example Jupyter Notebooks, visit: https://aka.ms/CXRReportGenDocs.

### Training information

| **Training Dataset** | **Details** |
Expand Down Expand Up @@ -129,4 +136,4 @@ For detailed guidelines on ethical use, refer to Microsoft’s [Responsible AI P
## Hardware Requirement for Compute Instances
- Supports CPU and GPU
- Default: Single A100 GPU or Intel CPU
- Minimum: Single GPU instance with 24Gb Memory (Fastest) or CPU
- Minimum: Single GPU instance with 24Gb Memory (Fastest) or CPU
7 changes: 6 additions & 1 deletion assets/models/system/medimageinsight/description.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
### Overview
Most medical imaging AI today is narrowly built to detect a small set of individual findings on a single modality like chest X-rays. This training approach is data- and computationally inefficient, requiring ~6-12 months per finding1, and often fails to generalize in real world environments. By further training existing multimodal foundation models on medical images and associated text data, Microsoft and Nuance created a multimodal foundation model that shows evidence of generalizing across various medical imaging modalities, anatomies, locations, severities, and types of medical data. The training methods learn to map the medical text and images into a unified numerical vector representation space, which makes it easy for computers to understand the relationships between those modalities.

Embeddings are an important building block in AI research and development for retrieval, search, comparison, classification, and tagging tasks, and developers and researchers can now use MedImageInsight embeddings in the medical domain. MedImageInsight embeddings is open source allowing developers to customize and adapt to their specific use cases.

This repository contains the MedImageInsight model, which is packaged in MLflow format and deployed using Azure ML service. The estimated time to package and deploy the model is approximately 1 hour.

This model is intended and provided as-is for research and model development exploration. MedImageInsight is not designed or intended to be deployed in clinical settings as-is nor is it for use in the diagnosis or treatment of any health or medical condition, and the model’s performance for such purposes has not been established.

You bear sole responsibility and liability for any use of MedImageInsight, including verification of outputs and incorporation into any product or service intended for a medical purpose or to inform clinical decision-making, compliance with applicable healthcare laws and regulations, and obtaining any necessary clearances or approvals.

Please see https://aka.ms/medimageinsightpaper for more details.

For documentation and example Jupyter Notebooks, visit: https://aka.ms/MedImageInsightDocs.
Expand Down
7 changes: 6 additions & 1 deletion assets/models/system/medimageparse/description.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
### Overview
Biomedical image analysis is fundamental for biomedical discovery in cell biology, pathology, radiology, and many other biomedical domains. MedImageParse is a biomedical foundation model for imaging parsing that can jointly conduct segmentation, detection, and recognition across 9 imaging modalities. Through joint learning, we can improve accuracy for individual tasks and enable novel applications such as segmenting all relevant objects in an image through a text prompt, rather than requiring users to laboriously specify the bounding box for each object.

MedImageParse is broadly applicable, performing image segmentation across 9 imaging modalities.
Expand All @@ -11,6 +10,12 @@ In summary, MedImageParse shows potential to be a building block for an all-in-o

It is broadly applicable to all major biomedical image modalities, which may pave a future path for efficient and accurate image-based biomedical discovery when built upon and integrated into an application.

This repository contains the MedImageParse model, which is packaged in MLflow format and deployed using Azure ML service. The estimated time to package and begin to build upon the model is approximately 1 hour.

This model is intended and provided as-is for research and model development exploration. MedImageParse is not designed or intended to be deployed in clinical settings as-is nor is it intended for use in the diagnosis or treatment of any health or medical condition, and the model’s performance for such purposes has not been established. You bear sole responsibility and liability for any use of MedImageParse, including verification of outputs and incorporation into any product or service intended for a medical purpose or to inform clinical decision-making, compliance with applicable healthcare laws and regulations, and obtaining any necessary clearances or approvals.

For documentation and example Jupyter Notebooks, visit: https://aka.ms/MedImageParseDocs.

### Model Architecture
MedImageParse is built upon a transformer-based architecture, optimized for processing large biomedical corpora. Leveraging multi-head attention mechanisms, it excels at identifying and understanding biomedical terminology, as well as extracting contextually relevant information from dense scientific texts. The model is pre-trained on vast biomedical datasets, allowing it to generalize across various biomedical domains with high accuracy.

Expand Down
2 changes: 1 addition & 1 deletion assets/models/system/phi-3.5-mini-128k-instruct/spec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -56,4 +56,4 @@ tags:
logging_strategy: "steps"
logging_steps: 10
save_total_limit: 1
version: 2
version: 3
2 changes: 1 addition & 1 deletion assets/models/system/phi-3.5-moe-128k-instruct/spec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,4 +55,4 @@ tags:
logging_strategy: "steps"
logging_steps: 10
save_total_limit: 1
version: 2
version: 3
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ RUN python -m nltk.downloader punkt
RUN python -m nltk.downloader punkt_tab
RUN MAX_JOBS=4 pip install flash-attn==2.5.5 --no-build-isolation

# dummy number to change when needing to force rebuild without changing the definition: 1
# dummy number to change when needing to force rebuild without changing the definition: 2
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ dependencies:
- python=3.9
- pip=22.1.2
- numpy~=1.22.3
- pandas~=1.3.5
- pandas~=1.5.3
- py-xgboost=1.3.3
- pyopenssl=24.2.1
- cryptography=42.0.2
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,7 @@ RUN pip install git+https://github.com/stanford-futuredata/megablocks.git@5897cd
# RUN pip install -e ./ --no-cache-dir

# When copied to assets repo, change to install from public pypi
RUN pip install llm-optimized-inference==0.2.12 --no-cache-dir

RUN pip uninstall transformers -y
RUN pip uninstall -y vllm
RUN pip install https://automlsamplenotebookdata.blob.core.windows.net/vllm/transformers-4.45.0.dev0-py3-none-any.whl
RUN pip install https://automlsamplenotebookdata.blob.core.windows.net/vllm/vllm-0.6.1.post2+cu124-cp38-abi3-manylinux1_x86_64.whl
RUN pip install llm-optimized-inference==0.2.14 --no-cache-dir

# clean conda and pip caches
RUN rm -rf ~/.cache/pip
Expand Down

0 comments on commit c71ab53

Please sign in to comment.