Страница телеграм канала Spark in me - Internet, data science, math, deep learning, philosophy
Spark in me - Internet, data science, math, deep learning, philosophy
1332 подписчиков
All this - lost like tears in rain. Internet, data science, math, deep learning, philosophy. No bs. Our website - http://spark-in.me Our chat - https://goo.gl/WRm93d DS courses review - http://goo.gl/5VGU5A - https://goo.gl/YzVUKf
Входит в категории: Технологии
20.09.2021 16:09
Embedding Quantization in PyTorch
Currently the following modules are supported by the following quantization modes:
- CNN (i.e. nn.Conv) => static quantization (you need to store statistics);
- RNN or Transformer (i.e. nn.Linear) => dynamic (you just convert weights);
Looks like less love was given to the Embedding layers, because they require static quantization: https://github.com/pytorch/pytorch/issues/65185
In any case, someone tested it and provided a simple recipe, which is absent from docs.
#deep_learning
07.09.2021 12:09
New German V4 Model and English V5 Models
New and improved models in Silero-models! Community edition versions available here: https://github.com/snakers4/silero-models
Huge performance improvements for two new models:
- English V5 (quality)
- German V3 (quality)
The models currently are available in the following flavors:
- English V5 jit (small), onnx (small), jit_q (small, quantized), jit_xlarge, onnx_xlarge
- German V3 jit_large, onnx_large
The xsmall model family for English in on the way.
The quality growth visualization:
03.09.2021 14:09
Digest 2021-08
Tech
Docker is Updating and Extending Our Product Subscriptions - https://www.docker.com/blog/updating-product-subscriptions/
New bubble - Looking to the skies: Funding for future air mobility takes off - https://www.mckinsey.com/industries/aerospace-and-defense/our-insights/future-air-mobility-blog/looking-to-the-skies-funding-for-future-air-mobility-takes-off
Ads, privacy and confusion - https://www.ben-evans.com/benedictevans/2021/8/27/understanding-privacy
I2P vs Tor - https://geti2p.net/en/comparison/tor, https://blokt.com/guides/what-is-i2p-vs-tor-browser
Hardware
Playing with LVM raids (RU) - https://aminux.wordpress.com/2021/08/24/lvm-over-soft-raid/
#digest
03.09.2021 14:09
Digest 2021-08
Datasets
Open Buildings
- https://sites.research.google/open-buildings/
- The dataset contains 516M building detections
- Area of 19.4M km2 (64% of the African continent)
The C4_200M Synthetic Dataset for Grammatical Error Correction
- https://ai.googleblog.com/2021/08/the-c4200m-synthetic-dataset-for.html
- conventional corruption model: begin with a grammatically correct sentence and then to “corrupt” it by adding error
- tagged corruption model that we propose builds: clean sentence as input + an error type tag that describes the kind of error
Two New Datasets for Conversational NLP: TimeDial and Disfl-QA
- https://ai.googleblog.com/2021/08/two-new-datasets-for-conversational-nlp.html
#digest
03.09.2021 14:09
Digest 2021-08
Blogs
AI has the worst superpower … medical racism - https://lukeoakdenrayner.wordpress.com/2021/08/02/ai-has-the-worst-superpower-medical-racism/
Why We Fund Teams And Hire Individuals - https://www.strangeloopcanon.com/p/why-we-fund-teams-and-hire-individuals
Enhanced support for citations on GitHub - https://github.blog/2021-08-19-enhanced-support-citations-github/
WDC Rearchitects the HDD - https://thessdguy.com/wdc-rearchitects-the-hdd/
Karma on Habr changes - https://habr.com/ru/company/habr/blog/575912/
#digest
03.09.2021 14:09
Digest 2021-08
Code
NumPy views: saving memory, leaking memory, and subtle bugs https://pythonspeed.com/articles/numpy-memory-views/
Multi-stage builds #1: Smaller images for compiled code - https://pythonspeed.com/articles/smaller-python-docker-images/
2021 SO developer survey - https://insights.stackoverflow.com/survey/2021
An Introduction to AI Story Generation - https://thegradient.pub/an-introduction-to-ai-story-generation/
A brief review of StyleGANs (RU) - https://habr.com/ru/company/recognitor/blog/568000/
The best Docker base image for your Python application (August 2021) - https://pythonspeed.com/articles/base-image-python-docker-images/
#digest
03.09.2021 14:09
Digest 2021-08
ML / Papers
Mapping Africa’s Buildings with Satellite Imagery - https://ai.googleblog.com/2021/07/mapping-africas-buildings-with.html
Machine Learning Wont Solve Natural Language Understanding - https://thegradient.pub/machine-learning-wont-solve-the-natural-language-understanding-challenge/
Police Misuse of Facial Recognition - Three Wrongful Arrests and Counting - https://www.skynettoday.com/briefs/wrongful-arrests
SoundStream: An End-to-End Neural Audio Codec - https://ai.googleblog.com/2021/08/soundstream-end-to-end-neural-audio.html
One weird trick to shrink convolutional networks for TinyML - https://petewarden.com/2021/08/05/one-weird-trick-to-shrink-convolutional-networks-for-tinyml/
qCT-Lung: Catching lung cancer early - https://blog.qure.ai/notes/qct-lung-catching-lung-cancer-early
Make Every feature Binary: A 135B parameter sparse neural network for massively improved search relevance - https://www.microsoft.com/en-us/research/blog/make-every-feature-binary-a-135b-parameter-sparse-neural-network-for-massively-improved-search-relevance/:
- over 200 billion binary features that reflect the subtle relationships between search queries and documents
- bing search
- LightGBM model plateau after 1 month of data
- This uses 3 years of search logs
- 9 billion features, generated from 49 feature groups, with each binary feature encoded into a 15-dimension embedding vector
- basically just a huge MLP model with huge embeddings
Improved Detection of Elusive Polyps via Machine Learning
- https://ai.googleblog.com/2021/08/improved-detection-of-elusive-polyps.html
- Trained on 3600 procedures (86M video frames) and tested it on 1400 procedures (33M frames)
- Detected 97% of the polyps (i.e., it yielded 97% sensitivity) at 4.6 false alarms per procedure
AI as a Hype Tool - https://www.skynettoday.com/editorials/ai-hype
#digest
03.09.2021 14:09
Digest 2021-08
Speech
Recreating Natural Voices for People with Speech Impairments - https://ai.googleblog.com/2021/08/recreating-natural-voices-for-people.html
#digest
03.09.2021 14:09
Digest 2021-08
I will continue to publish digests in bite-sized chunks in telegram, mostly links that pierce the noise, separated by the topic.
#digest
27.08.2021 20:08
https://medium.com/@zaharchikishev/tensorflow-sad-story-cf8e062d84ba
TF still is awesome? Have not touched for years.
Please share your experience - is this article truthful?