Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fill olc gap #1006

Open
grossir opened this issue Apr 17, 2024 · 2 comments
Open

Fill olc gap #1006

grossir opened this issue Apr 17, 2024 · 2 comments

Comments

@grossir
Copy link
Contributor

grossir commented Apr 17, 2024

Part of #929
We have 0 documents between January 16th, 2020 and July 6th, 2021. Missing 16 documents

Also missing this document for range 2013-10-21 to 2014-06-16

@grossir grossir moved this to In Progress in @grossir's backlog Apr 17, 2024
grossir added a commit to grossir/juriscraper that referenced this issue Apr 17, 2024
@grossir
Copy link
Contributor Author

grossir commented Apr 17, 2024

Commands to fill the gap:

docker exec -it cl-django python /opt/courtlistener/manage.py cl_back_scrape_opinions  --courts juriscraper.opinions.united_states.administrative_agency.olc --backscrape-start=01/16/2020 --backscrape-end=07/06/2021`
docker exec -it cl-django python /opt/courtlistener/manage.py cl_back_scrape_opinions  --courts juriscraper.opinions.united_states.administrative_agency.olc --backscrape-start=10/21/2013 --backscrape-end=06/16/2014`

@grossir
Copy link
Contributor Author

grossir commented Jun 3, 2024

After running the backscraper, we found that the opinions are actually in the DB, but they do not appear when searching. The CL search returns no result, but all the opinions from that time period exist in the DB and surface when searching by hash

The opinions do not appear when searching with court_id: olc because the docket for all these documents is assigned to court ag

[{'name': 'Exclusion of Religiously Affiliated Schools from Charter-School Grant Program',
  'url': 'https://www.justice.gov/olc/file/1347781/dl',
  'date': '2020/02/18',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=379b2f21780a0dd2181c550e2c23662d4dc6e11c'},
 {'name': 'Applicability of Section 410 of the Amtrak Reform and Accountability Act of 1997 to the Gateway Development Commission',
  'url': 'https://www.justice.gov/olc/file/1277456/dl',
  'date': '2020/02/13',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=73a96c0f86bb8da2142b3f8985afceab7e8a1327'},
 {'name': "House Committees' Authority to Investigate for Impeachment",
  'url': 'https://www.justice.gov/olc/file/1236346/dl',
  'date': '2020/01/19',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=26f179c84bb00e5c67d0baf04ca7066b882c8ab1'},
 {'name': 'Publication of a Report to the President on the Effect of Automobile and Automobile-Part Imports on the National Security',
  'url': 'https://www.justice.gov/olc/file/1350151/dl',
  'date': '2020/01/17',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=1870afdc264f7b0f36c088b2ca21323028c4de2d'},
 {'name': 'Congressional Oversight of the White House',
  'url': 'https://www.justice.gov/olc/file/1355831/dl',
  'date': '2021/01/08',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=0e4f0456738e8e1ffbcfdf530ed8e16b9ff2d1e6'},
 {'name': 'Executive Branch Participation in the Cyberspace Solarium Commission',
  'url': 'https://www.justice.gov/olc/file/1355826/dl',
  'date': '2020/10/09',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=6d66c6a5ab8632533a3d4641b6bb284691fca62a'},
 {'name': "Reimbursing the Attorney's Fees of Current and Former Federal Employees Interviewed as Witnesses in the Mueller Investigation",
  'url': 'https://www.justice.gov/olc/file/1347786/dl',
  'date': '2020/10/07',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=a4da5977dbcdb6210400673a843613927bf46174'},
 {'name': 'Congressionally Mandated Notice Period for Withdrawing from the Open Skies Treaty',
  'url': 'https://www.justice.gov/olc/file/1348136/dl',
  'date': '2020/09/22',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=5fe787d09c16096d2ac2861aae0ab045d6c52cda'},
 {'name': 'Authority of the Special Inspector General for Pandemic Recovery to Oversee Programs Established Under the CARES Act',
  'url': 'https://www.justice.gov/olc/file/1390936/dl',
  'date': '2021/04/29',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=bd1702719b910cfa0cd759c3ce52c833e4eb41c8'},
 {'name': 'Preemption of State and Local Requirements Under a PREP Act Declaration',
  'url': 'https://www.justice.gov/olc/file/1356956/dl',
  'date': '2021/01/19',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=e9d75d45bd2c6160c1f352a0c61664f10a72d41f'},
 {'name': 'Military Support for Customs and Border Protection Along the Southern Border Under the Posse Comitatus Act',
  'url': 'https://www.justice.gov/olc/file/1357086/dl',
  'date': '2021/01/19',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=f878c6c02be31074ea2b3349003ab6be1b4d517d'},
 {'name': 'Application of the Hyde Amendment to Federal Student-Aid Programs',
  'url': 'https://www.justice.gov/olc/file/1355946/dl',
  'date': '2021/01/16',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=5f3f3e73a154ffa6fd859d18ba6e611db82ee78e'},
 {'name': 'Responsibility for Electronic Presidential Records on Hardware of the Executive Office of the President After a Presidential Transition',
  'url': 'https://www.justice.gov/olc/file/1355841/dl',
  'date': '2021/01/15',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=f3b590c595c84c01a53359e80d474f669b36bd14'},
 {'name': 'Religious Seasonal Decorations in Federal Government Buildings',
  'url': 'https://www.justice.gov/olc/file/1355846/dl',
  'date': '2021/01/15',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=08ec2f4533b0b62d9d989ae4731f306463eb120a'},
 {'name': 'Home Confinement of Federal Prisoners After the COVID-19 Emergency',
  'url': 'https://www.justice.gov/olc/file/1355886/dl',
  'date': '2021/01/15',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=a1991d6f0b8d1884f94a84fb2fa323b8d8c07451'},
 {'name': 'Sex Segregation in Youth Rodeo Events Under Title IX Regulations',
  'url': 'https://www.justice.gov/olc/file/1355836/dl',
  'date': '2021/01/13',
  'sha1': 'https://www.courtlistener.com/api/rest/v3/opinions/?sha1=a725ad8efffbfce06c062d461734ebd99ca7d325'}]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

No branches or pull requests

1 participant