Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Readme: Brought "Knowledge" ahead of "Skills" #1311

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

rfrht
Copy link

@rfrht rfrht commented Oct 4, 2024

For someone training their models, the most common use case is creating knowledge - and not skills. Knowledge QNAs have strict requirements, needs schema version 3 - and an interested party can be misled with the "skills" examples.

This commit brings Knowledge QNA ahead of Skills, highlight the difference between them - and also highlights the context and qna pair requirements.

signed-off-by: [email protected]

For someone training their models, the most **common use case** is creating knowledge - and not skills. Knowledge QNAs have strict requirements, needs schema version 3 - and an interested party can be misled with the "skills" examples.

This commit brings Knowledge QNA ahead of Skills, highlight the difference between them - and also highlights the context and qna pair requirements.

Signed-off-by: Rodrigo Freire <[email protected]>
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Oct 4, 2024
@jjasghar
Copy link
Member

jjasghar commented Oct 4, 2024

Thank you for this, but we are attempting to migrate to docs.instructlab.ai. This rewrite maybe worth looking at that repo instead

@rfrht
Copy link
Author

rfrht commented Oct 4, 2024

Thank you for this, but we are attempting to migrate to docs.instructlab.ai. This rewrite maybe worth looking at that repo instead

Hello, Asghar! Thanks for your review. I'll be definitely doing it - already forked it and will propose the changes over there too.

Meanwhile, can you approve this PR for this repo, hoping to help our users? I took issue on the thing because of the amount of time that I lost figuring out the qna :-)

Best regards,

  • RF.

QNAs have a requirement of *exactly* five contexts, with *exactly* three Q/A pairs. Reinforcing the verbiage.

Signed-off-by: Rodrigo Freire <[email protected]>
@rfrht
Copy link
Author

rfrht commented Oct 7, 2024

Added another commit to this batch - Another important one clarifying the requirements for Q&A block: I learnt the hard way that the Q&A file requires exactly 5 contexts, with exactly 3 Q/A pairs.

Reinforcing the verbiage.

CC: @mairin

@jjasghar
Copy link
Member

jjasghar commented Oct 8, 2024

After continued discussion at the triage meeting, please make these changes at the docs.instructlab.ai, we want to cut down these readmes (across the project) and centralize non-developer content at docs.instructlab.ai.

@bjhargrave
Copy link
Contributor

I learnt the hard way that the Q&A file requires exactly 5 contexts, with exactly 3 Q/A pairs

I am curious about this. I have heard it before but could not find anywhere in the sdg code where this was enforced or required. Do you have a pointer? If this is truly the case, then the schema should also enforce this. Thanks.

@bjhargrave
Copy link
Contributor

For someone training their models, the most common use case is creating knowledge - and not skills.

I am not sure this is true for all. It may be true for you in the use cases you are working on. I have seen other cases where it was almost all compositional skills with little to no knowledge.

Knowledge and compositional skills are both relevant but one is not more important than another.

@bjhargrave
Copy link
Contributor

Knowledge QNAs have strict requirements, needs schema version 3

Compositional skills also need schema version 3 when contributing to this repo.

@rfrht
Copy link
Author

rfrht commented Oct 11, 2024

Hello @bjhargrave

Compositional skills also need schema version 3 when contributing to this repo.

Alright, the existing documentation in README.md shows schema Version 2 for skills, all around.

Where can I find the proper Schema V3 for Skill, so I can update this content?

Best,

  • RF.

@bjhargrave
Copy link
Contributor

V3 Compositional Skills schema is the same schema as for V2. So the only change needed is to update the version to 3.

The schemas are at https://github.com/instructlab/schema/tree/main/src/instructlab/schema

* Updated required Skills schema version to 3
* Improved documentation on how to specify files in Knowledge files

Signed-off-by: Rodrigo Freire <[email protected]>
@rfrht
Copy link
Author

rfrht commented Oct 11, 2024

Hello, BJ!

Added a new commit to the batch. Check the diff here and the final rendered page here.

@jjasghar
Copy link
Member

I really think we should be cutting this README down and moving to http://github.com/instructlab/docs.instructlab.ai for any of these changes.

Our readmes are so much, and confusing, and any effort here will just be eventually moved over anyway.

@rfrht
Copy link
Author

rfrht commented Oct 16, 2024

I wholeheartedly agree with that. This Readme should be wiped out, have some few words explaining what it is about and then referring to the new documentation URL.

Meanwhile, I think that we are sitting on the content and people are consuming outdated content.

Copy link

github-actions bot commented Nov 1, 2024

This pull request has been automatically marked as stale because it has not had activity within 15 days. It will be automatically closed if no further activity occurs within the next 31 days.

@github-actions github-actions bot added the stale stale-bot has marked you as stale label Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation stale stale-bot has marked you as stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants