Skip to content

Commit

Permalink
Randomness, Instagram View Stories, and more (#29)
Browse files Browse the repository at this point in the history
* Randomness

* fix gatsby/react console errors

* Randomness Docs Polish

* deleteIndexedDB doc

* getCookies, deleteCookies docs

* Cookies & Scrapers docs

* indexedDB, input, random docs

- added missing helpers

* instagram nav, auth, stories docs

* homepage polish

* overview polish

* install polish

* aborting polish

* removing 'export' from code displayed

* instagram botactions designed at desktop width

- adjust example to set Viewport to desktop width to fix viewStories
  • Loading branch information
mrWh1te authored Apr 21, 2021
1 parent bf2fe54 commit b338eb1
Show file tree
Hide file tree
Showing 15 changed files with 502 additions and 107 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ export const Container = (props) => {
<a href="https://www.npmjs.com/org/botmation" target="_blank" rel="noreferrer" aria-label="View @botmation packages on NPM" style={{position: 'relative', right: '120px'}} >
<NPMLogo />
</a>
<a href="https://github.com/mrWh1te/Botmation" class="github-corner" aria-label="View source on GitHub" target="_blank" rel="noreferrer">
<a href="https://github.com/mrWh1te/Botmation" className="github-corner" aria-label="View source on GitHub" target="_blank" rel="noreferrer">
<GithubLogo />
</a>
</ExtraLogosContainer>
Expand Down
13 changes: 6 additions & 7 deletions src/@rocketseat/gatsby-theme-docs/text/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,19 @@ description: 'Build web bots declaratively in TypeScript with the new simple fra
[<img src="https://david-dm.org/mrWh1te/Botmation/status.svg" alt="" />](https://david-dm.org/mrWh1te/Botmation)
![GitHub](https://img.shields.io/github/license/mrWh1te/Botmation)

[Botmation](https://github.com/mrWh1te/Botmation) is a functional TypeScript framework for declaratively building bots with [Puppeteer](https://github.com/puppeteer/puppeteer). It maximizes code *readability*, *maintainability* and *testability* with its compositional design.
[Botmation](https://github.com/mrWh1te/Botmation) is a TypeScript framework for declaratively building bots with [Puppeteer](https://github.com/puppeteer/puppeteer). It maximizes code *readability*, *maintainability* and *testability* with its compositional design.

These bots can do whatever people do with a web browser and more. They can automate social media, take screenshots, make PDF's, scrape, surf the web and much more. 🌊🏄📸
These bots can do whatever people do with a web browser and more. They can surf and scrape complex web applications, automate social media, take screenshots, generate website PDF's, and much more! 🌊🏄📸

## Features

<ul style="padding-left:.75rem;list-style:none;">
<li>🤓 Learn only what you need</li>
<li>🛣️ Simple pattern to follow</li>
<li>🥇 100% Core library test coverage</li>
<li>📚 Get started with Functional Programming</li>
<li>🤓 Learn only what you need</li>
<li>🛣️ Follow one simple coding pattern</li>
<li>⚡️ Write readable, testable & composable code</li>
<li>🏢 Use higher order functions</li>
<li>🧱 Assemble complex async functionality</li>
<li>🧱 Assemble complex async functionality with ease</li>
<li>🥇 Be confident with 100% Core library test coverage</li>
<li>🔥 and much more!</li>
</ul>

Expand Down
14 changes: 8 additions & 6 deletions src/config/sidebar.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,18 @@

- label: 'Topics'
items:
- label: 'Aborting'
link: '/topic/aborting'
- label: 'Piping'
link: '/topic/piping'
- label: 'Injects'
link: '/topic/injects'
- label: 'Conditionals'
link: '/topic/conditionals'
- label: 'Loops'
link: '/topic/loops'
- label: 'Error Handling'
link: '/topic/error-handling'
- label: 'Aborting'
link: '/topic/aborting'
- label: 'Piping'
link: '/topic/piping'
- label: 'Injects'
link: '/topic/injects'
- label: 'Concurrency'
link: '/topic/concurrency'

Expand Down Expand Up @@ -67,6 +67,8 @@
link: '/api/navigation'
- label: 'Pipe'
link: '/api/pipe'
- label: 'Random'
link: '/api/random'
- label: 'Scrapers'
link: '/api/scrapers'
- label: 'Time'
Expand Down
64 changes: 55 additions & 9 deletions src/docs/api/cookies.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,27 +2,26 @@
title: 'Cookies'
---

These higher order functions provide a simple way to load and save cookies from and to a Puppeteer page to a local JSON file.
These BotActions provide the means to manage cookies from a Puppeteer page.

You can configure where the cookie JSON files are saved & loaded by using the `BotFileOptions` param as either an inject or as a higher-order param. Higher-order `BotFileOptions` values override injected `BotFileOptions` values.
`BotFileOptions` is used to specify the directory for saving and loading cookies, relative to the bot's executing directory.

These functions are compatible with the higher-order [files()() BotAction](/api/files#files) to inject a customized `BotFileOptions` into each assembled BotAction.
These functions are compatible with the higher-order [files()() BotAction](/api/files#files) to inject a customized `BotFileOptions` into each assembled BotAction. That allows you to standardize the directory for all assembled BotActions that are reading and writing files.

## Save Cookies
Saves the cookies from the Puppeteer page in a JSON file with the name provided.

```typescript
const saveCookies = (fileName: string, botFileOptions?: Partial<BotFileOptions>): BotFilesAction => async(page, options: Partial<BotFileOptions>) => {
const hydratedOptions = enrichBotFileOptionsWithDefaults({...options, ...botFileOptions})

const cookies = await page.cookies()
await fs.writeFile(getFileUrl(hydratedOptions.cookies_directory, hydratedOptions, fileName) + '.json', JSON.stringify(cookies, null, 2))
}
```
`BotFileOptions` is optional, in all cases (higher-order param, as an inject). If nothing is provided, the default is the local directory.

TODO: link vv
See [enrichBotFileOptionsWithDefaults]() for details on setting the directory for files read and created.
See [enrichBotFileOptionsWithDefaults](/api/files#enrichbotfileoptionswithdefaults) for details on setting the directory for files read and created.

For an usage example, see the [Instagram example](https://github.com/mrWh1te/Botmation/blob/master/src/examples/instagram.ts), where saving and loading cookies enables the Bot to skip the login flow on subsequent runs, as long as the cookies saved, have not yet expired.

Expand All @@ -43,7 +42,54 @@ const loadCookies = (fileName: string, botFileOptions?: Partial<BotFileOptions>)
```
`BotFileOptions` is optional, in all cases (higher-order param, as an inject). If nothing is provided, the default is the local directory.

TODO: link vv
See [enrichBotFileOptionsWithDefaults]() for details on setting the directory for files read and created.
See [enrichBotFileOptionsWithDefaults](/api/files#enrichbotfileoptionswithdefaults) for details on setting the directory for files read and created.

For an usage example, see the [Instagram example](https://github.com/mrWh1te/Botmation/blob/master/src/examples/instagram.ts), where saving and loading cookies enables the Bot to skip the login flow on subsequent runs, as long as the cookies saved, have not yet expired.

## Get Cookies

This BotAction returns all cookies for the inputted URL's. When no URL's are specified, it will return the cookies for the current URL only.

```typescript
const getCookies = (...urls: string[]): BotAction<Protocol.Network.Cookie[]> => async(page) =>
page.cookies(...urls)
```

## Delete Cookies

This BotAction deletes all cookies provided.

```typescript
const deleteCookies = (...cookies: Protocol.Network.Cookie[]): BotAction => async(page, ...injects) => {
if (cookies.length === 0) {
if (injectsHavePipe(injects)) {
const pipeValue = getInjectsPipeValue(injects)
if (Array.isArray(pipeValue) && pipeValue.length > 0) {
cookies = pipeValue
}
}
}

if (cookies.length > 0) {
return page.deleteCookie(...cookies)
}
}
```

It's intended to be paired with [getCookies](#getcookies) to designate which cookies are to be deleted. For example:

```typescript
pipe()(
getCookies(),
deleteCookies()
)(page)
```
This will grab all cookies, for the current URL, then pipe those cookies into `deleteCookies` for deletion. Otherwise, specify the cookies via the higher-order parameter:

```typescript
chain(
deleteCookies(cookie1, cookie2, cookie3)
)(page)
```

For an usage example, see the [Instagram example](https://github.com/mrWh1te/Botmation/blob/master/src/examples/instagram.ts), where saving and loading cookies enables the Bot to skip the login flow on subsequent runs, as long as the cookies saved, have not yet expired.
> If both higher-order params and pipe values are used, this will, like every other BotAction, use the higher-order param instead
47 changes: 45 additions & 2 deletions src/docs/api/indexed-db.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -104,11 +104,28 @@ const getIndexedDBValue =
For an usage example, see the [Instagram isGuest](/sites/instagram#isguest-conditionalbotaction) BotAction.

## Delete Indexed DB

This BotAction attempts to delete an IndexedDB database, provided the database name. It's possible for this to fail, you can read more about the used underlying logic [here](https://developer.mozilla.org/en-US/docs/Web/API/IDBFactory/deleteDatabase).

```typescript
const deleteIndexedDB = (databaseName?: string): BotIndexedDBAction<void> => async(page, ...injects) => {
const [, , injectDatabaseName] = unpipeInjects(injects, 2)

await page.evaluate(
deleteIndexedDBDatabase,
databaseName ?? injectDatabaseName
)
}
```

This BotAction is compatible with the [indexedDBStore()()](#indexeddbstore) for injecting the name of the database for deletion.

## Helpers

In order to interact with IndexedDB in the Puppeteer Page, separate functions need to be "evaluated" in the Page. Therefore, the following Helper functions are setup as if running inside a Browser Page, instead of NodeJS.
In order to interact with IndexedDB in the Puppeteer Page, separate functions are serialized and evaluated within the browser page. To accomplish this, the following Helper functions were created to be evaluated in the web pages.

> If you `console.log()` in an evaluated Page function, don't look for it in NodeJS's console, but the Puppeteer Page's console.
> If you `console.log()` in an evaluated Page function, the result will appear inside the browser's Console window and *not* the NodeJS terminal.
### setIndexedDBStoreValue()

Expand Down Expand Up @@ -181,4 +198,30 @@ function getIndexedDBStoreValue(databaseName: string, databaseVersion: number, s
}
})
}
```

### deleteIndexedDBDatabase()

```typescript
function deleteIndexedDBDatabase(databaseName: string) {
return new Promise<void>((resolve, reject) => {
const DBDeleteRequest = indexedDB.deleteDatabase(databaseName);

DBDeleteRequest.onerror = function(event) {
logError('delete IndexedDB Database name = ' + databaseName)
event.stopPropagation()
return reject(this.error)
};

DBDeleteRequest.onsuccess = function() {
return resolve()
};

DBDeleteRequest.onblocked = function(event) {
event.stopPropagation()
logMessage('blocked attempt to delete IndexedDB Database name = ' + databaseName)
return resolve()
};
})
}
```
41 changes: 36 additions & 5 deletions src/docs/api/input.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,36 @@ title: 'Input'
These BotAction's provide ways to input into a page as User.

## Click
Does a left-mouse click on a HTML element by the provided HTML selector.
Does a left-mouse click on the first HTML element that matches the provided HTML selector.

```typescript
const click = (selector: string): BotAction => async(page) =>
await page.click(selector)
```
Example:
Example:
```typescript
await chain(
click('form input[type="submit"]'),
waitForNavigation
)(page)
```

## Click Text
Does a left-mouse click on the first element found whose text content equals the text provided.

```typescript
const clickText = (text: string): BotAction =>
evaluate(clickElementWithText, text)
```

Example:
```typescript
chain(
clickText('Save Info')
)(page)
```
This will click the DOM element with the text "Save Info".

## Type
Type with an imaginary "keyboard" the copy provided.

Expand All @@ -27,10 +43,25 @@ const type = (copy: string): BotAction => async(page) =>
await page.keyboard.type(copy)
```

Example:
Example:
```typescript
await chain(
click('form input[name="username"]'),
type('my-username')
click('form input[name="email"]'),
type('[email protected]')
)(page)
```

## Helpers

### clickElementWithText

```typescript
const clickElementWithText = (text: string) => {
const xpath = `//*[text()='${text}']`
const matchingElement = document.evaluate(xpath, document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;

if (matchingElement instanceof HTMLElement) {
matchingElement.click()
}
}
```
7 changes: 2 additions & 5 deletions src/docs/api/navigation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,13 @@ These BotAction's provide simple ways to change the Page's URL.
## Go To
Navigates the page to the url provided, unless the active url is the url provided, then it emits a warning in the console.
```typescript
const goTo = (url: string, goToOptions?: Partial<DirectNavigationOptions>): BotAction =>
const goTo = (url: string, goToOptions?: Partial<WaitForOptions>): BotAction =>
async(page) => {
goToOptions = enrichGoToPageOptions(goToOptions)

// same url check
if (page.url() === url) {
return
}

await page.goto(url, goToOptions)
await page.goto(url, enrichGoToPageOptions(goToOptions))
}
```
`goToOptions` are provided aa they are enriched with safe defaults. See Puppeteer's [Documentation on page.goto()](https://devdocs.io/puppeteer/#pagegotourl-options) for details.
Expand Down
Loading

0 comments on commit b338eb1

Please sign in to comment.