
GITHUB . COM {
}
Detected CMS Systems:
- Wordpress (2 occurrences)
Title:
PERF: groupby on an unsorted index slows to a crawl. works fine if index is sorted. Β· Issue #46527 Β· pandas-dev/pandas
Description:
Pandas version checks I have checked that this issue has not already been reported. I have confirmed this issue exists on the latest version of pandas. I have confirmed this issue exists on the main branch of pandas. Reproducible Example...
Website Age:
17 years and 8 months (reg. 2007-10-09).
Matching Content Categories {π}
- Technology & Computing
- Health & Fitness
- Cryptocurrency
Content Management System {π}
What CMS is github.com built with?
Github.com utilizes WORDPRESS.
Traffic Estimate {π}
What is the average monthly size of github.com audience?
ππ Tremendous Traffic: 10M - 20M visitors per month
Based on our best estimate, this website will receive around 13,918,967 visitors per month in the current month.
check SE Ranking
check Ahrefs
check Similarweb
check Ubersuggest
check Semrush
How Does Github.com Make Money? {πΈ}
Subscription Packages {π³}
We've located a dedicated page on github.com that might include details about subscription plans or recurring payments. We identified it based on the word pricing in one of its internal links. Below, you'll find additional estimates for its monthly recurring revenues.How Much Does Github.com Make? {π°}
Subscription Packages {π³}
Prices on github.com are in US Dollars ($).
They range from $4.00/month to $21.00/month.
We estimate that the site has approximately 6,945,396 paying customers.
The estimated monthly recurring revenue (MRR) is $29,170,665.
The estimated annual recurring revenues (ARR) are $350,047,976.
Wordpress Themes and Plugins {π¨}
What WordPress theme does this site use?
It is strange but we were not able to detect any theme on the page.
What WordPress plugins does this website use?
It is strange but we were not able to detect any plugins on the page.
Keywords {π}
index, issue, pandas, tickers, groupby, unsorted, sorted, performance, sign, perf, slows, crawl, works, fine, projects, closed, furechan, indexed, added, navigation, code, pull, requests, actions, security, version, confirmed, exists, import, ncols, sample, rawdata, runs, triage, reviewed, team, member, memory, execution, speed, lukemanley, jreback, milestone, github, type, footer, skip, content, menu, product,
Topics {βοΈ}
personal information perf comment metadata assignees np import pandas unsorted index slows latest version type projects projects milestone 1 issue exists triage issue import numpy rawdata = rawdata pandas works fine main branch tickers = tickers[ 5 closed 100% complete relationships pd nkeys indexed = rawdata github issue perf columns=columns sorted runs index groupby np code nkeys sample data columns= runs sign indexed sorted skip jump crawl checked reported confirmed reproducible nrows ncols = 50 %04d range ncols dataframe zeros
Payment Methods {π}
- Braintree
Questions {β}
- Already have an account?
Schema {πΊοΈ}
DiscussionForumPosting:
context:https://schema.org
headline:PERF: groupby on an unsorted index slows to a crawl. works fine if index is sorted.
articleBody:### Pandas version checks
- [X] I have checked that this issue has not already been reported.
- [X] I have confirmed this issue exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas.
- [X] I have confirmed this issue exists on the main branch of pandas.
### Reproducible Example
import numpy as np
import pandas as pd
nkeys, nrows, ncols = 50, 5000, 10
tickers = ["X%04d" % i for i in range(nkeys)]
columns = ["C%d" % i for i in range(ncols)]
sample = pd.DataFrame(np.zeros((nrows, ncols)), columns=columns)
tickers = tickers[::-1] # to reverse the tickers
data = {t: sample for t in tickers}
rawdata = pd.concat(data, names=["ticker"])
rawdata = rawdata.reset_index().drop(columns="level_1")
indexed = rawdata.set_index('ticker')
indexed.groupby('ticker').apply(lambda x:x)
### Installed Versions
<details>
INSTALLED VERSIONS
------------------
commit : 06d230151e6f18fdb8139d09abf539867a8cd481
python : 3.8.8.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.22000
machine : AMD64
processor : Intel64 Family 6 Model 158 Stepping 13, GenuineIntel
byteorder : little
LC_ALL : en_US.UTF-8
LANG : en_US.UTF-8
LOCALE : English_United States.1252
pandas : 1.4.1
numpy : 1.20.1
pytz : 2021.1
dateutil : 2.8.1
pip : 21.0.1
setuptools : 52.0.0.post20210125
Cython : 0.29.23
pytest : 6.2.3
hypothesis : None
sphinx : 4.0.1
blosc : None
feather : None
xlsxwriter : 1.3.8
lxml.etree : 4.6.3
html5lib : 1.1
pymysql : 1.0.2
psycopg2 : None
jinja2 : 2.11.3
IPython : 7.22.0
pandas_datareader: 0.10.0
bs4 : 4.9.3
bottleneck : 1.3.2
fastparquet : None
fsspec : 0.9.0
gcsfs : None
matplotlib : 3.3.4
numba : 0.53.1
numexpr : 2.7.3
odfpy : None
openpyxl : 3.0.7
pandas_gbq : None
pyarrow : 7.0.0
pyreadstat : None
pyxlsb : None
s3fs : 0.4.2
scipy : 1.6.2
sqlalchemy : 1.4.7
tables : 3.6.1
tabulate : None
xarray : None
xlrd : 2.0.1
xlwt : 1.3.0
zstandard : None
</details>
### Prior Performance
When indexed is sorted runs under 100 ms.
When not indexed (ie ticker as column) also runs under 100 ms.
Just remove the following code to check :
tickers = tickers[::-1]
author:
url:https://github.com/furechan
type:Person
name:furechan
datePublished:2022-03-27T13:57:55.000Z
interactionStatistic:
type:InteractionCounter
interactionType:https://schema.org/CommentAction
userInteractionCount:0
url:https://github.com/46527/pandas/issues/46527
context:https://schema.org
headline:PERF: groupby on an unsorted index slows to a crawl. works fine if index is sorted.
articleBody:### Pandas version checks
- [X] I have checked that this issue has not already been reported.
- [X] I have confirmed this issue exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas.
- [X] I have confirmed this issue exists on the main branch of pandas.
### Reproducible Example
import numpy as np
import pandas as pd
nkeys, nrows, ncols = 50, 5000, 10
tickers = ["X%04d" % i for i in range(nkeys)]
columns = ["C%d" % i for i in range(ncols)]
sample = pd.DataFrame(np.zeros((nrows, ncols)), columns=columns)
tickers = tickers[::-1] # to reverse the tickers
data = {t: sample for t in tickers}
rawdata = pd.concat(data, names=["ticker"])
rawdata = rawdata.reset_index().drop(columns="level_1")
indexed = rawdata.set_index('ticker')
indexed.groupby('ticker').apply(lambda x:x)
### Installed Versions
<details>
INSTALLED VERSIONS
------------------
commit : 06d230151e6f18fdb8139d09abf539867a8cd481
python : 3.8.8.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.22000
machine : AMD64
processor : Intel64 Family 6 Model 158 Stepping 13, GenuineIntel
byteorder : little
LC_ALL : en_US.UTF-8
LANG : en_US.UTF-8
LOCALE : English_United States.1252
pandas : 1.4.1
numpy : 1.20.1
pytz : 2021.1
dateutil : 2.8.1
pip : 21.0.1
setuptools : 52.0.0.post20210125
Cython : 0.29.23
pytest : 6.2.3
hypothesis : None
sphinx : 4.0.1
blosc : None
feather : None
xlsxwriter : 1.3.8
lxml.etree : 4.6.3
html5lib : 1.1
pymysql : 1.0.2
psycopg2 : None
jinja2 : 2.11.3
IPython : 7.22.0
pandas_datareader: 0.10.0
bs4 : 4.9.3
bottleneck : 1.3.2
fastparquet : None
fsspec : 0.9.0
gcsfs : None
matplotlib : 3.3.4
numba : 0.53.1
numexpr : 2.7.3
odfpy : None
openpyxl : 3.0.7
pandas_gbq : None
pyarrow : 7.0.0
pyreadstat : None
pyxlsb : None
s3fs : 0.4.2
scipy : 1.6.2
sqlalchemy : 1.4.7
tables : 3.6.1
tabulate : None
xarray : None
xlrd : 2.0.1
xlwt : 1.3.0
zstandard : None
</details>
### Prior Performance
When indexed is sorted runs under 100 ms.
When not indexed (ie ticker as column) also runs under 100 ms.
Just remove the following code to check :
tickers = tickers[::-1]
author:
url:https://github.com/furechan
type:Person
name:furechan
datePublished:2022-03-27T13:57:55.000Z
interactionStatistic:
type:InteractionCounter
interactionType:https://schema.org/CommentAction
userInteractionCount:0
url:https://github.com/46527/pandas/issues/46527
Person:
url:https://github.com/furechan
name:furechan
url:https://github.com/furechan
name:furechan
InteractionCounter:
interactionType:https://schema.org/CommentAction
userInteractionCount:0
interactionType:https://schema.org/CommentAction
userInteractionCount:0
External Links {π}(3)
Analytics and Tracking {π}
- Site Verification - Google
Libraries {π}
- Clipboard.js
- D3.js
- Lodash
Emails and Hosting {βοΈ}
Mail Servers:
- aspmx.l.google.com
- alt1.aspmx.l.google.com
- alt2.aspmx.l.google.com
- alt3.aspmx.l.google.com
- alt4.aspmx.l.google.com
Name Servers:
- dns1.p08.nsone.net
- dns2.p08.nsone.net
- dns3.p08.nsone.net
- dns4.p08.nsone.net
- ns-1283.awsdns-32.org
- ns-1707.awsdns-21.co.uk
- ns-421.awsdns-52.com
- ns-520.awsdns-01.net