
GITHUB . COM {
}
Detected CMS Systems:
- Wordpress (2 occurrences)
Title:
Private model training (DP-SGD) with sparse features Β· Issue #1370 Β· WICG/turtledove
Description:
Hello, Private model training has been recently mentioned here. One of the privacy considerations is to include DP in the training loop through DP-SGD. There are cases when DP-SGD would make the training process considerably slower as it...
Website Age:
17 years and 8 months (reg. 2007-10-09).
Matching Content Categories {π}
- Education
- Photography
- Virtual Reality
Content Management System {π}
What CMS is github.com built with?
Github.com employs WORDPRESS.
Traffic Estimate {π}
What is the average monthly size of github.com audience?
ππ Tremendous Traffic: 10M - 20M visitors per month
Based on our best estimate, this website will receive around 10,000,019 visitors per month in the current month.
However, some sources were not loaded, we suggest to reload the page to get complete results.
check SE Ranking
check Ahrefs
check Similarweb
check Ubersuggest
check Semrush
How Does Github.com Make Money? {πΈ}
Subscription Packages {π³}
We've located a dedicated page on github.com that might include details about subscription plans or recurring payments. We identified it based on the word pricing in one of its internal links. Below, you'll find additional estimates for its monthly recurring revenues.How Much Does Github.com Make? {π°}
Subscription Packages {π³}
Prices on github.com are in US Dollars ($).
They range from $4.00/month to $21.00/month.
We estimate that the site has approximately 4,989,889 paying customers.
The estimated monthly recurring revenue (MRR) is $20,957,532.
The estimated annual recurring revenues (ARR) are $251,490,385.
Wordpress Themes and Plugins {π¨}
What WordPress theme does this site use?
It is strange but we were not able to detect any theme on the page.
What WordPress plugins does this website use?
It is strange but we were not able to detect any plugins on the page.
Keywords {π}
training, dpsgd, sign, private, model, features, techniques, projects, issue, sparsity, navigation, open, pull, requests, actions, security, sparse, lufe, privacy, make, case, aware, research, topic, considered, csharrison, github, labels, type, milestone, footer, skip, content, menu, product, solutions, resources, source, enterprise, pricing, search, jump, wicg, turtledove, public, notifications, fork, star, code, issues,
Topics {βοΈ}
private model training protected audience api comment metadata assignees type projects projects milestone privacy considerations training loop categorical features dp-sgd recently mentioned include dp gradients calculated rendering impossible embedding tables explainer linked org/abs/2404 exploring supporting specific approaches assigned labels labels type milestone relationships research phase github preserve sparsity optimization techniques features research sparsity techniques skip jump sign cases make destroys backprop rely case working llms aware topic remedy clear considered context face thoughts feedback concern
Payment Methods {π}
- Braintree
Questions {β}
- Already have an account?
- Are there any techniques that are being considered to face this or thoughts about this topic?
- Did you have any specific approaches you wanted us to consider?
Schema {πΊοΈ}
DiscussionForumPosting:
context:https://schema.org
headline:Private model training (DP-SGD) with sparse features
articleBody:Hello,
Private model training has been recently mentioned [here](https://github.com/WICG/turtledove/blob/main/PA_private_model_training.md). One of the privacy considerations is to include DP in the training loop through DP-SGD.
There are cases when DP-SGD would make the training process considerably slower as it destroys the sparsity of the gradients calculated during backprop, rendering impossible to use optimization techniques that rely on such sparsity. This is usually the case when some features are categorical features or working with embedding tables in the case of LLMs. I am aware there is research around this topic to remedy it, although it is not clear from the explainer linked above if this is something that has been considered in the context of Protected Audience API.
Are there any techniques that are being considered to face this or thoughts about this topic?
Thanks
author:
url:https://github.com/Lufe44
type:Person
name:Lufe44
datePublished:2024-12-17T23:53:36.000Z
interactionStatistic:
type:InteractionCounter
interactionType:https://schema.org/CommentAction
userInteractionCount:1
url:https://github.com/1370/turtledove/issues/1370
context:https://schema.org
headline:Private model training (DP-SGD) with sparse features
articleBody:Hello,
Private model training has been recently mentioned [here](https://github.com/WICG/turtledove/blob/main/PA_private_model_training.md). One of the privacy considerations is to include DP in the training loop through DP-SGD.
There are cases when DP-SGD would make the training process considerably slower as it destroys the sparsity of the gradients calculated during backprop, rendering impossible to use optimization techniques that rely on such sparsity. This is usually the case when some features are categorical features or working with embedding tables in the case of LLMs. I am aware there is research around this topic to remedy it, although it is not clear from the explainer linked above if this is something that has been considered in the context of Protected Audience API.
Are there any techniques that are being considered to face this or thoughts about this topic?
Thanks
author:
url:https://github.com/Lufe44
type:Person
name:Lufe44
datePublished:2024-12-17T23:53:36.000Z
interactionStatistic:
type:InteractionCounter
interactionType:https://schema.org/CommentAction
userInteractionCount:1
url:https://github.com/1370/turtledove/issues/1370
Person:
url:https://github.com/Lufe44
name:Lufe44
url:https://github.com/Lufe44
name:Lufe44
InteractionCounter:
interactionType:https://schema.org/CommentAction
userInteractionCount:1
interactionType:https://schema.org/CommentAction
userInteractionCount:1
External Links {π}(3)
Analytics and Tracking {π}
- Site Verification - Google
Libraries {π}
- Clipboard.js
- D3.js
- Lodash
Emails and Hosting {βοΈ}
Mail Servers:
- aspmx.l.google.com
- alt1.aspmx.l.google.com
- alt2.aspmx.l.google.com
- alt3.aspmx.l.google.com
- alt4.aspmx.l.google.com
Name Servers:
- dns1.p08.nsone.net
- dns2.p08.nsone.net
- dns3.p08.nsone.net
- dns4.p08.nsone.net
- ns-1283.awsdns-32.org
- ns-1707.awsdns-21.co.uk
- ns-421.awsdns-52.com
- ns-520.awsdns-01.net