Cannot access a private page with custom unofficialNotionAPI argument

[jshingler]

Hi, ... Got the Private Pages Info, ... Thanks.

Does the graph navigation handle Circular Links between pages.

When I run the app, ... it just runs and runs. It never finishes.

[9oelM]

Do you mind sharing your code? I have tested with a test page where I created circular links between two pages, but it still works. You also might want to juggle some optional configs defined by Typescript.

Here's my page that acts as the root node: https://lizard-flax-e38.notion.site/Test-bc95525429244d169cad7f6725cb5b34?pvs=4

Running the scraper gives me the following result, without hanging:

{
  "nodes": [
    {
      "id": "bc955254-2924-4d16-9cad-7f6725cb5b34",
      "title": "Test",
      "type": "page",
      "spaceId": "fda5cc70-a0d3-4f12-9b33-2ec0c26ab6ca",
      "parentId": "none",
      "cc": 2
    },
    {
      "id": "65786275-82a8-4c32-8bd7-11d49a3ff7c6",
      "parentId": "bc955254-2924-4d16-9cad-7f6725cb5b34",
      "spaceId": "fda5cc70-a0d3-4f12-9b33-2ec0c26ab6ca",
      "type": "page",
      "title": "A"
    },
    {
      "id": "175b9592-7cf5-4452-be34-da896731654b",
      "parentId": "bc955254-2924-4d16-9cad-7f6725cb5b34",
      "spaceId": "fda5cc70-a0d3-4f12-9b33-2ec0c26ab6ca",
      "type": "page",
      "title": "B"
    }
  ],
  "links": [
    {
      "source": "65786275-82a8-4c32-8bd7-11d49a3ff7c6",
      "target": "bc955254-2924-4d16-9cad-7f6725cb5b34"
    },
    {
      "source": "65786275-82a8-4c32-8bd7-11d49a3ff7c6",
      "target": "175b9592-7cf5-4452-be34-da896731654b"
    },
    {
      "source": "175b9592-7cf5-4452-be34-da896731654b",
      "target": "bc955254-2924-4d16-9cad-7f6725cb5b34"
    }
  ],
  "errors": []
}

[jshingler]

I used the code as is from the info you gave me, .... the page id is the blog.
And I used the project from the example scapper and replaced index.js with this code

import { NotionGraph } from "@graphcentral/notion-graph-scraper"
import fs from "fs"
import { NotionAPI } from "notion-client";

/**

• example of how to use @graphcentral/notion-graph-scraper
  */
  ;(async () => {
  const notionApi = new NotionAPI({
  activeUser: process.env.NOTION_ACTIVE_USER,
  authToken: process.env.NOTION_TOKEN_V2
  })
  const notionGraph = new NotionGraph({
  maxDiscoverableNodes: 10,
  maxDiscoverableNodesInOtherSpaces: 10,
  verbose: true,
  unofficialNotionAPI: notionApi
  })
  const graph = await notionGraph.buildGraphFromRootNode(
  // Some random Japanese blog
  '95fcfe03257541c5aaa21dd43bdbc381'
  )
  console.log(graph.nodes.length)
  console.log(graph.links.length)
  await new Promise((resolve, reject) => {
  fs.writeFile(test0.json, JSON.stringify(graph), (err) => {
  if (err) reject(err)
  else resolve(``)
  })
  });

process.exit(0)
})()

[9oelM]

@jshingler but you configured process.env.NOTION_ACTIVE_USER and process.env.NOTION_TOKEN_V2 right?
what is the output when you run it?

[9oelM]

and the japanese blog should be accessible without using the custom notionApi argument because it's already public

[jshingler]

Yes I Configured the environment variables, ....
I am not using the Japanese Blog, ... I just didn't want to expose my private page id

Here is a portion of the output, it just keeps repeating

[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0
[12/4/2023, 9:44:24 PM] # current requests: 0 / # items in the queue: 0
[12/4/2023, 9:44:24 PM] # total requests sent: 0
[12/4/2023, 9:44:24 PM] # total successful requests: 0

[9oelM]

Yeah if it keeps showing zeroes, it shouldn't be about circular links because there's no page to start with. There should be something about not being able to retrieve the information in the first place. Let me try with my own private pages and get back to you

[jshingler]

Looking at this again tonight, ... here is the error message that shows up in test0.json.

d.map is not a function. in notion-utils/build/index.js

{"links":[],"nodes":[],"errors":[{"name":"TypeError","message":"d.map is not a function","stack":"TypeError: d.map is not a function\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4072)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)\n at s (file:///Users/jshingler/Archive/Workspaces/graphcentral/notion-scrape-example/node_modules/notion-utils/build/index.js:1:4308)"},{}]}

[jshingler]

It works on VERY simple private pages, .... Seems to be React-Notion-X has issues with something on the more complex pages.