GITHUB . COM {}

Detected CMS Systems:

Wordpress (2 occurrences)

Analyzed Page
Matching Content Categories
CMS
Monthly Traffic Estimate
How Does Github.com Make Money
How Much Does Github.com Make
Wordpress Themes And Plugins
Keywords
Topics
Payment Methods
Questions
Schema
External Links
Analytics And Tracking
Libraries
Hosting Providers

We are analyzing https://github.com/googleapis/nodejs-datastore/issues/95.

Title:
Errors importing lots of entities to DataStore using the emulator · Issue #95 · googleapis/nodejs-datastore
Description:
From @glenpike on May 23, 2018 10:26 [x] - Search the issues already opened: https://github.com/GoogleCloudPlatform/google-cloud-node/issues [x] - Search StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform+node...
Website Age:
17 years and 9 months (reg. 2007-10-09).

Matching Content Categories {📚}

Technology & Computing
Mobile Technology & AI
Business & Finance

Content Management System {📝}

What CMS is github.com built with?

Github.com is based on WORDPRESS.

Traffic Estimate {📈}

What is the average monthly size of github.com audience?

🚀🌠 Tremendous Traffic: 10M - 20M visitors per month

Based on our best estimate, this website will receive around 10,666,394 visitors per month in the current month.

check SE Ranking
check Ahrefs
check Similarweb
check Ubersuggest
check Semrush

How Does Github.com Make Money? {💸}

Subscription Packages {💳}

We've located a dedicated page on github.com that might include details about subscription plans or recurring payments. We identified it based on the word pricing in one of its internal links. Below, you'll find additional estimates for its monthly recurring revenues.

How Much Does Github.com Make? {💰}

Subscription Packages {💳}

Prices on github.com are in US Dollars ($). They range from $4.00/month to $21.00/month.
We estimate that the site has approximately 5,347,507 paying customers.
The estimated monthly recurring revenue (MRR) is $22,459,531.
The estimated annual recurring revenues (ARR) are $269,514,368.

Wordpress Themes and Plugins {🎨}

What WordPress theme does this site use?

It is strange but we were not able to detect any theme on the page.

What WordPress plugins does this website use?

It is strange but we were not able to detect any plugins on the page.

Keywords {🔍}

error, import, datastore, code, cancelled, received, rststream, emulator, issue, request, justinbeckwith, internal, halfclosed, memory, errors, information, data, commented, issues, stephenplusplus, sign, importing, lots, entities, glenpike, thread, type, size, api, search, version, googlecloudcommon, const, deadlineexceeded, info, unable, ideasculptor, ohmpatel, increase, googleapisnodejsdatastore, navigation, pull, requests, actions, security, closed, opened, check, save, works,

Topics {✒️}

/questions/tagged/google-cloud-platform+node googleapis/nodejs-datastore api /googlecloudplatform/google-cloud-node/issues [ 0 google-cloud-node version errors importing lots datastore issues related osx el capitan entity types constituting xms512m -xmx1152m -xx maxpermsize=256m -xx maxnewsize=256m -xms total dataset size 826z error import 827z error import 841z error import 858z error import 859z error import 873z error import 874z error import 898z error import 899z error import 145z error import importing data 1gb including indexes basic import command troubleshooting guide lots docs eventually errors oom errors native thread [datastore] total size small fraction error code 8 ideasculptor edits ohmpatel1997 edits issues [datastore] exception google cloud ] - search js version 2 npm version half-closed native method storage bucket persist datastore datastore service errors original issue

Payment Methods {📊}

Braintree

Questions {❓}

5k models, we are getting lots of errors that look like they're coming from grpc maybe?
@glenpike does this repo using the datastore service, and not the emulator?
@stephenplusplus could I trouble you to take a look?
Already have an account?
Any ideas?
Are there any workarounds?

Schema {🗺️}

DiscussionForumPosting:
      context:https://schema.org
      headline:Errors importing lots of entities to DataStore using the emulator
      articleBody:_From @glenpike on May 23, 2018 10:26_ [x] - Search the issues already opened: https://github.com/GoogleCloudPlatform/google-cloud-node/issues [x] - Search StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform+node.js [404] - Check our Troubleshooting guide: https://googlecloudplatform.github.io/google-cloud-node/#/docs/guides/troubleshooting [404] - Check our FAQ: https://googlecloudplatform.github.io/google-cloud-node/#/docs/guides/faq If you are still having issues, please be sure to include as much information as possible: #### Environment details - gcloud SDK: 202.0.0 - OS: OSX El Capitan (10.11.6) Using about 12GB / 16GB memory - Node.js version: v8.11.2 - npm version: v5.6.0 - google-cloud-node version: ``` ├─┬ @google-cloud/[email protected] │ ├─┬ @google-cloud/[email protected] ├─┬ @google-cloud/[email protected] │ └─┬ @google-cloud/[email protected] │ ├─┬ @google-cloud/[email protected] │ ├─┬ @google-cloud/[email protected] ├─┬ @google-cloud/[email protected] │ ├─┬ @google-cloud/[email protected] ``` Using DataStore via: [email protected] #### Steps to reproduce Looping through a list of data and creating a [model](https://sebelga.gitbooks.io/gstore-node/content/entity/creation.html) for each one, then calling a function which uses [save](https://sebelga.gitbooks.io/gstore-node/content/entity/methods/save.html) ``` const { body } = ctx; let promises = []; // Save new Object body.new.forEach((model) => { const createdEntity = FromModel.create(model, model.id); promises.push(createdEntity.upsert()); }); const response = await Promise.all(promises); ``` Trying to import about 2.5k models, we are getting lots of errors that look like they're coming from grpc maybe? Workaround is to split data into chunks, e.g. 1/4 works. The log of errors looks like this ('...' is replacing several repeated events): ``` 10:02:46.826Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.826Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.827Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 ... 10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.841Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.841Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 ... 10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.858Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.858Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.858Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.859Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.873Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.874Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 ... 10:02:46.898Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.898Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.898Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.898Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.899Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.899Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:03:43.145Z ERROR import: 4 DEADLINE_EXCEEDED: Deadline Exceeded 10:03:43.145Z ERROR import: 4 DEADLINE_EXCEEDED: Deadline Exceeded ``` The DEADLINE_EXCEEDED error seems to correspond with this in the emulator: ``` datastore] May 23, 2018 11:03:16 AM com.google.cloud.datastore.emulator.impl.LocalDatastoreFileStub$7 run [datastore] INFO: Time to persist datastore: 198 ms [datastore] Exception in thread "LocalDatastoreService-1" java.lang.OutOfMemoryError: unable to create new native thread [datastore] at java.lang.Thread.start0(Native Method) [datastore] at java.lang.Thread.start(Thread.java:714) [datastore] at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950) [datastore] at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1018) [datastore] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) [datastore] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [datastore] at java.lang.Thread.run(Thread.java:745) [datastore] Exception in thread "LocalDatastoreService-4" java.lang.OutOfMemoryError: unable to create new native thread [datastore] at java.lang.Thread.start0(Native Method) [datastore] at java.lang.Thread.start(Thread.java:714) [datastore] at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950) [datastore] at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1018) [datastore] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) [datastore] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [datastore] at java.lang.Thread.run(Thread.java:745) ``` _Copied from original issue: GoogleCloudPlatform/google-cloud-node#2822_
      author:
         url:https://github.com/JustinBeckwith
         type:Person
         name:JustinBeckwith
      datePublished:2018-05-31T21:08:01.000Z
      interactionStatistic:
         type:InteractionCounter
         interactionType:https://schema.org/CommentAction
         userInteractionCount:6
      url:https://github.com/95/nodejs-datastore/issues/95
      context:https://schema.org
      headline:Errors importing lots of entities to DataStore using the emulator
      articleBody:_From @glenpike on May 23, 2018 10:26_ [x] - Search the issues already opened: https://github.com/GoogleCloudPlatform/google-cloud-node/issues [x] - Search StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform+node.js [404] - Check our Troubleshooting guide: https://googlecloudplatform.github.io/google-cloud-node/#/docs/guides/troubleshooting [404] - Check our FAQ: https://googlecloudplatform.github.io/google-cloud-node/#/docs/guides/faq If you are still having issues, please be sure to include as much information as possible: #### Environment details - gcloud SDK: 202.0.0 - OS: OSX El Capitan (10.11.6) Using about 12GB / 16GB memory - Node.js version: v8.11.2 - npm version: v5.6.0 - google-cloud-node version: ``` ├─┬ @google-cloud/[email protected] │ ├─┬ @google-cloud/[email protected] ├─┬ @google-cloud/[email protected] │ └─┬ @google-cloud/[email protected] │ ├─┬ @google-cloud/[email protected] │ ├─┬ @google-cloud/[email protected] ├─┬ @google-cloud/[email protected] │ ├─┬ @google-cloud/[email protected] ``` Using DataStore via: [email protected] #### Steps to reproduce Looping through a list of data and creating a [model](https://sebelga.gitbooks.io/gstore-node/content/entity/creation.html) for each one, then calling a function which uses [save](https://sebelga.gitbooks.io/gstore-node/content/entity/methods/save.html) ``` const { body } = ctx; let promises = []; // Save new Object body.new.forEach((model) => { const createdEntity = FromModel.create(model, model.id); promises.push(createdEntity.upsert()); }); const response = await Promise.all(promises); ``` Trying to import about 2.5k models, we are getting lots of errors that look like they're coming from grpc maybe? Workaround is to split data into chunks, e.g. 1/4 works. The log of errors looks like this ('...' is replacing several repeated events): ``` 10:02:46.826Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.826Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.827Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 ... 10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.841Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.841Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.841Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 ... 10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.858Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.858Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.858Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.858Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.859Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.873Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.874Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 ... 10:02:46.898Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.898Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.898Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.898Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:02:46.899Z ERROR import: 13 INTERNAL: Half-closed without a request 10:02:46.899Z ERROR import: 1 CANCELLED: Received RST_STREAM with error code 8 10:03:43.145Z ERROR import: 4 DEADLINE_EXCEEDED: Deadline Exceeded 10:03:43.145Z ERROR import: 4 DEADLINE_EXCEEDED: Deadline Exceeded ``` The DEADLINE_EXCEEDED error seems to correspond with this in the emulator: ``` datastore] May 23, 2018 11:03:16 AM com.google.cloud.datastore.emulator.impl.LocalDatastoreFileStub$7 run [datastore] INFO: Time to persist datastore: 198 ms [datastore] Exception in thread "LocalDatastoreService-1" java.lang.OutOfMemoryError: unable to create new native thread [datastore] at java.lang.Thread.start0(Native Method) [datastore] at java.lang.Thread.start(Thread.java:714) [datastore] at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950) [datastore] at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1018) [datastore] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) [datastore] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [datastore] at java.lang.Thread.run(Thread.java:745) [datastore] Exception in thread "LocalDatastoreService-4" java.lang.OutOfMemoryError: unable to create new native thread [datastore] at java.lang.Thread.start0(Native Method) [datastore] at java.lang.Thread.start(Thread.java:714) [datastore] at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950) [datastore] at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1018) [datastore] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) [datastore] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [datastore] at java.lang.Thread.run(Thread.java:745) ``` _Copied from original issue: GoogleCloudPlatform/google-cloud-node#2822_
      author:
         url:https://github.com/JustinBeckwith
         type:Person
         name:JustinBeckwith
      datePublished:2018-05-31T21:08:01.000Z
      interactionStatistic:
         type:InteractionCounter
         interactionType:https://schema.org/CommentAction
         userInteractionCount:6
      url:https://github.com/95/nodejs-datastore/issues/95
Person:
      url:https://github.com/JustinBeckwith
      name:JustinBeckwith
      url:https://github.com/JustinBeckwith
      name:JustinBeckwith
InteractionCounter:
      interactionType:https://schema.org/CommentAction
      userInteractionCount:6
      interactionType:https://schema.org/CommentAction
      userInteractionCount:6

External Links {🔗}(7)

Analytics and Tracking {📊}

Site Verification - Google

Libraries {📚}

Clipboard.js
D3.js
GSAP
Lodash

Emails and Hosting {✉️}

Mail Servers:

aspmx.l.google.com
alt1.aspmx.l.google.com
alt2.aspmx.l.google.com
alt3.aspmx.l.google.com
alt4.aspmx.l.google.com

Name Servers:

dns1.p08.nsone.net
dns2.p08.nsone.net
dns3.p08.nsone.net
dns4.p08.nsone.net
ns-1283.awsdns-32.org
ns-1707.awsdns-21.co.uk
ns-421.awsdns-52.com
ns-520.awsdns-01.net

9.57s.