How to use Puppeteer with YouTube/YouTube Studio

Question

I am trying to automate a process in YouTube Studio with Puppeteer to share/unshare private YouTube videos with a certain email address through a Node.js REST API backend server written in TypeScript.

I have created two endpoints in the server:

1. /share - For sharing YouTube Videos
2. /unshare - For un-sharing YouTube Videos.

I first started with the /share endpoint controller.

My thought process was as follows:

1. First login into Google
2. Redirect to YouTube Studio
3. Go into the content section of the channel, and search for the videos
4. If no videos were found, then the server will send back to the client a response that no videos were found. If the search was success-full, and it found videos, then it should loop through the videos, open a new tab to open the video details page and update the private email list.

Unfortunately, I could not select the visibility dropdown of the video in the videos page, because it is not an HTML element. Google created YouTube Studio with Polymer and created custom tags. This means, that if I wanted to query select a custom youtube element with puppeteer with await page.$('custom-youtube-selector') it will throw an exception that the element does not exist in the DOM or it is not a HTML element.

Video Visibility Select Button

I found that Puppeteer did add support for query selecting Shadow DOM elements last year however there is nothing written in their documentation. In addition, I used a custom Query Handler library called query-selector-shadow-dom however when I tried to query select the element it return null.

Down below I am adding two repos. The first repo is written in TypeScript, the second is written in JavaScript (query-selector-shadow-dom did not work well with TypeScript compiler).

1. TypeScript Repo - https://github.com/almog-gutin/YouTube-Sharing-Automation-Server-Node-TS
2. JavaScript Repo - https://github.com/almog-gutin/YouTube-Sharing-Automation-Server-Node-JS

Answer 1

It doesn't seem to be documented at present, but there's a pierce/ handler for querying within shadow roots. See the related GitHub PR and associated test example in the current version of Puppeteer (13.1.2).

I've adapted the example in this comment below, so that you can reproduce it to see that it works. I've also written a helper function for you to use when querying within shadow roots.

TS Playground

// Adapted from https://github.com/puppeteer/puppeteer/issues/858#issuecomment-441429302

import {default as puppeteer} from 'puppeteer';

/**
 * For use when querying within shadow roots.
 *
 * Ref: https://github.com/puppeteer/puppeteer/pull/6509
 *
 * ```ts
 * // const className = 'action';
 * const button = await page.$<HTMLButtonElement>(pierce`#elementWithinAShadowRoot > button.${className}`);
 * ```
 */
function pierce (
  strings: TemplateStringsArray,
  ...stringExpressions: string[]
): string {
  const mutable = [...strings];
  let selector = 'pierce/';
  while (mutable.length > 0) {
    selector += `${mutable.shift() ?? ''}${stringExpressions.shift() ?? ''}`;
  }
  return selector;
}

(async () => {
  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();
  const url = 'https://npm-demos.appspot.com/@polymer/[email protected]/demo/index.html';
  await page.goto(url, {waitUntil: 'networkidle2'});
  const input = await page.$<HTMLInputElement>(pierce`#input-2 > input`);
  if (!input) throw new Error('Input not found');
  await input.focus();
  await input.type('woof woof!');
  await page.screenshot({path: './typing.png', type: 'png'});
  await browser.close();
})();

// Check the saved screenshot "./typing.png" to see the typed string in the input