He's a good adventure dude.
586 stories
·
6 followers

WTF: Signal Adds Cryptocurrency Support

3 Comments and 10 Shares

According to Wired, Signal is adding support for the cryptocurrency MobileCoin, “a form of digital cash designed to work efficiently on mobile devices while protecting users’ privacy and even their anonymity.”

Moxie Marlinspike, the creator of Signal and CEO of the nonprofit that runs it, describes the new payments feature as an attempt to extend Signal’s privacy protections to payments with the same seamless experience that Signal has offered for encrypted conversations. “There’s a palpable difference in the feeling of what it’s like to communicate over Signal, knowing you’re not being watched or listened to, versus other communication platforms,” Marlinspike told WIRED in an interview. “I would like to get to a world where not only can you feel that when you talk to your therapist over Signal, but also when you pay your therapist for the session over Signal.”

I think this is an incredibly bad idea. It’s not just the bloating of what was a clean secure communications app. It’s not just that blockchain is just plain stupid. It’s not even that Signal is choosing to tie itself to a specific blockchain currency. It’s that adding a cryptocurrency to an end-to-end encrypted app muddies the morality of the product, and invites all sorts of government investigative and regulatory meddling: by the IRS, the SEC, FinCEN, and probably the FBI.

And I see no good reason to do this. Secure communications and secure transactions can be separate apps, even separate apps from the same organization. End-to-end encryption is already at risk. Signal is the best app we have out there. Combining it with a cryptocurrency means that the whole system dies if any part dies.

Read the whole story
kleer001
105 days ago
reply
This is stupid and it will die quickly. There's far too much friction on so many levels. Again, a mature software starts adding things nobody is asking for. Hopefully whomever green lit this will feel bad and any more stupid ideas will be slow in coming.
popular
109 days ago
reply
Share this story
Delete
1 public comment
jepler
113 days ago
reply
"adding a cryptocurrency to an end-to-end encrypted app muddies the morality of the product, and invites all sorts of government investigative and regulatory meddling: by the IRS, the SEC, FinCEN, and probably the FBI."

"Signal is the best app we have out there. Combining it with a cryptocurrency means that the whole system dies if any part dies"
Earth, Sol system, Western spiral arm

Decoding The “Bitcoin Full Node Sculpture,” A Cypherpunk Chronometer

1 Comment

Bidding for the artwork covered in this story will be open during a 24-hour, bitcoin-only auction on February 5, 2021 at 3:33 p.m. EST.

YouTube Video

The Bitcoin Full Node Sculpture is a cypherpunk chronometer, designed to work as a fully-functioning bitcoin full node and block explorer. Modeled after the ancient navigation tool, the astrolabe, it acts as a compass for the humble bitcoiner to set to true north.

Visualizing the clock-and-calendar-like nature of Bitcoin stretching over 132 years, the sculpture represents a digital metronome ticking endlessly in the halls of time and space. This cyber-metronome ticks at a steady pace of “every 10 minutes,” as this is in perfect harmony with Bitcoin time. Like the Bitcoin software itself, the sculpture is a tool for personal growth, a tool that connects us to the community, a full circle. Also like the Bitcoin software, the design evolves with each new version.

The sculpture represents a snapshot of a moment in Bitcoin time. In Version 4.0, the node is set to Bitcoin block height 630,000. This is a significant block height in Bitcoin history. Block 630,000 is the first block of the third Halving epoch. It’s the first block with a 6.25 BTC per block “block subsidy” and the first block ever in the history of Bitcoin in the “single digit” reward era.

Image for post

I’m going to break this article into three sections:

  1. The Vision
  2. The Node Architecture
  3. The Future

There’s a lot here, so if you’re ready to sit back and take a trip down the Bitcoin rabbit hole, let’s get started!

Section 1: The Vision

I wanted to do something to help Bitcoin. I believe that open and decentralized projects can only grow through community input. If I’m in the community, I want to contribute. I’m an artist, so my contribution would be to bring beauty into the world, to try to create something new, something meaningful, that would stand the test of time.

While the idea of “Bitcoin as a cosmic timekeeping device” came to me quickly, bringing it from thought to reality took over seven months of extremely intense work. The redesign for “Number 4” took an additional five months of work. I learned the true meaning of proof of work at a personal level, and I’m so thankful for the opportunity to have gone on this journey and create something meaningful to me. In life, I think that’s the best we can hope for, healthy, wholesome, honest work, a fulfilling journey and rewarding results.

Section 2: The Architecture Of The Bitcoin Full Node Sculpture

First, we’ll take a look at the design schematics, then we’ll break it down into 21 “blocks” (or layers) and dive into the stories contained within each ring.

Image for post
Image for post

Each of the layers displays specific information, and we’ll need this key to decode the inner workings of the node. We’ll go through each of the 21 layers together. You’ll see the number 21 encoded within the mandala several times and in different ways, this is all, of course, in reference to the hard limit on the number of bitcoin(s) that can ever exist, 21 million.

Image for post
Refer back to this map as we take our trip down the rabbit hole, don’t get lost!

Layer 1: Bitcoin Block Stats; Current Network State

You’ll notice that Layer 1 is broken into two distinct sections, a yellow outer ring and an orange inner ring. The outer ring is our constant, this doesn’t change; the inner ring shows the current network state as of the most recent block mined. The “stats” displayed in the inner, orange part of Layer 1 update every 10 minutes as the node validates each new block. (The most recent block is called the “chain tip” and this is the block data displayed in Layer 1.)

Let’s take a look at the block stats for bitcoin block height 630,000:

Image for post
Each block has a unique set of data, displayed by your node.
There’s also some Bitcoin “ethos” encoded at the very top center of Ring 1.

Each time a new block is found, Layer 1 updates to display the current network info (current block height, block date and timestamp, network difficulty, hash rate, transaction-specific details and the current version of Bitcoin Core that the node is running (0.19.0 was the most current release as of May 11, 2020, when block 630,000 was mined).

Layer 2: The Wheel Of Time

Layer 2 is composed of 132 years, divided into 34 sections of four years each. The four-year cycles within the wheel of time track the Bitcoin Halvings.

“The number of bitcoins generated per block is set to decrease geometrically, with a 50% reduction every 210,000 blocks, or approximately four years”

Bitcoin Wiki

The four-year cycles are shown in a geometrically decreasing pattern: Year one of a Halving cycle is the smallest, and the fourth year is the tallest.

Image for post
Note how 2009 is the smallest and 2012 the tallest in the four-year cycle.

Notice the euler marks below the “Years Containers”? The larger ruler marks measure the four-year Halving cycles, by year (pictured in yellow below), and the smaller ruler marks below those (in pink/purple below) track the difficulty adjustments. Each ruler mark represents 10 difficulty adjustments (there are about 26 per year, so 2.6 ruler marks per year below).

Image for post
Yellow tracks calendar years, purple tracks difficulty adjustments.

Layer 3: Bitcoin Issuance By Halving

The cut-out sections in the node architecture represent the bitcoin issuance over the 34 Halving cycles. In the first Halving cycle, we see that 10,500,000 BTC (50 percent of all bitcoin that will ever exist), are mined in that first Halving cycle. The largest cut out section represents the period from 2009 to 2012 when the block reward was 50 BTC issued every 10 minutes. You can see, clear as day, the compression of the issuance over time. Time is not a kind mistress here.

Image for post
Image for post

Pay close attention to the size of the cut-out sections. These represent the portion of bitcoin’s total issuance per each four-year Halving cycle

At the top of the ring is the “Halving Crest” emblazoned with the mathematical formula that describes the halving function in the bitcoin source code. Interesting to note that the source code references 64 halvings, yet we reach 0 BTC per block after 33.

Image for post
The Halving Crest
Image for post
Bitcoin source code
Image for post

Layer 4: Binary Encoded Data

Like all good software, there are Easter Eggs to be found and decoded both in this sculpture and Bitcoin as an organism. Ring four is tiny, almost hidden, but for those who look deeper, there’s always more to find. I tried a few fun Bitcoin phrases but in the end, the yellow highlighted strings were used.

Image for post

I would like to link you to the source for the top reference, where you can learn more about why this was used. “Bitcoin: Everything There Is, Divided By 21 million,” a piece by Knut Svenholm (you’ll also see reference to another work of his, “Sovereignty Through Mathematics,” a wonderful book of which you can read the first chapter for free. You can listen to the full audio here (also free).

If you want to learn more about Faketoshi and his mad-cap antics, this is a must-follow Twitter account.

Layer 5: The Merkle Tree Layer

Image for post

This is an important ring and it covers a lot of ground. Merkle trees are critically important to the way Bitcoin operates. You’ve heard of “the blockchain,” well, this is how it works.

The layer is composed of 21 Merkle trees divided into three sections of seven each (since 21 is a triangular number). There are 18 “blocks” linking the Merkle trees to each other, all the way back to genesis. Each Bitcoin block is unique, like the faces on quartz crystals, etched into form by time and energy, never to change. To capture this uniqueness, the 18 “blocks” aren’t the standard cube shape you’d expect to represent a block. The 18 blocks are represented by the Platonic and Archimedean solids, the building blocks of all matter in the universe. The 18 solids plus the three “Crests” (or “Pillars” of Bitcoin), 18 plus three equals 21.

Image for post
Here are a few of the Archimedean solids with the mathematics of the edge/face/vertices noted.
Image for post
Here we see a hidden triangle, as 21 is a triangular number.

The three crests, at the points of the triangle above represent the “Three Pillars of Bitcoin.”

  • Social
  • Technical
  • Financial

As a student of Bitcoin, I try to educate myself in all the different areas of importance, to ensure my education is well rounded. This well-rounded education is a foundation to build upon. I find that to dive deeply into Bitcoin, I must immerse myself into those three aspects. In each of the three pillars, I try to highlight aspects which to me are personally important, but I’m sure I am missing some of your favorites. Let’s go through the sub-subjects included in each section of this ring.

  • The Social Pillar: This represents the community aspects of Bitcoin. The peer-to-peer nature of it, the game theory and politics, even environmental impacts and benefits.
Image for post
  • The Legend of Satoshi: Here we have the story of an anonymous cypherpunk hacker who drops a white paper on a cryptography mailing list in 2008, who exhibits perfect OpSec and stays around just long enough to bootstrap the world’s first digital decentralized financial network.
  • The Immaculate Conception: Bitcoin’s origin story cannot be replicated. In Gigi’s fifth of his 21 lessons, he states:

“What might look like a weird anonymity stunt is actually crucial for a truly decentralized system. No centralized control. No centralized authority. No inventor. No one to prosecute, torture, blackmail or extort. An immaculate conception of technology.”

  • Open Source: Bitcoin is free and open-source software
  • Decentralized: This one doesn’t need much explanation, but let’s link here to Stratum V2, the next step in decentralizing mining pools
  • Peer-To-Peer: Bitcoin does not need third parties or mediators. It allows us to connect to our neighbors near and far in a direct network, connected to one another. What more could the world want?
  • Eco Friendly: This has been one of the more controversial inclusions, due to misinformation about Bitcoin’s energy use, however I’m steadfast in my belief that it should be included. Make sure to read section four on Bitcoin’s energy use in the Stone Ridge 2020 Shareholder Letter;

“As Bitcoin finances the for-profit development of cheap, clean energy infrastructure on a massive scale, it can lead to a future in which more and more of the world’s population lives near abundant energy with an extraordinarily low marginal cost of production. This matters because cheap energy equals human flourishing. That’s an equation. Cheap energy = human flourishing.

“Beyond the revolution in monetary policy that Bitcoin already represents, Bitcoin may also represent the biggest catalyst the world has ever known for developing abundant, clean, cheap energy. And, therefore, one of biggest catalysts in the world for human flourishing.”

  • The Technical Pillar: “What’s under the hood?” How does this thing work?
Image for post
You’ll find the equation for bitcoin ECDSA, Secp256k1 at the top of the crest
Image for post
  • Merkle Trees: Merkle trees act as a “source of truth” in Bitcoin because blocks can be verified from the chain tip, all the way back to the Genesis Block. “Leaves of the Merkle tree used in bitcoin are typically hashes of single blocks.” Merkle tree hashes can be compared by full nodes to verify the integrity of the chain, and detect any inconsistencies.
  • ECDSA: This is how bitcoin addresses, and “public/private key pairs” can be generated, transactions signed, etc. See here: “Elliptic Curve Digital Signature Algorithm
  • SHA-256: According to Bitcoin Wiki, “SHA-256 is a member of the SHA-2 cryptographic hash functions designed by the NSA. SHA stands for Secure Hash Algorithm. Cryptographic hash functions are mathematical operations run on digital data; by comparing the computed ‘hash’ (the output from execution of the algorithm) to a known and expected hash value, a person can determine the data’s integrity. A one-way hash can be generated from any piece of data, but the data cannot be generated from the hash.”
  • The Difficulty Adjustment: An astonishing mechanism for computational fairness. Using blocks, not time, the network self-adjusts. Bitcoin, in essence, is winding its own watch! “Every two weeks (or, more precisely, every 2,016 blocks), the bitcoin protocol adjusts the difficulty of finding blocks.” Bitcoin seeks to create a harmonic resonance with blocks mined at a frequency of once every 10 minutes. If blocks are coming too fast, difficulty adjusts up, and blocks become harder to find. If blocks are coming too slow, difficulty adjusts down. A mind-boggling mechanism that works regardless of how much, or how little compute-power is directed at the network.
  • The Financial Pillar: Let’s investigate some of the monetary properties of bitcoin that make it the best money the world’s ever seen.
Image for post
  • Digital Scarcity: Completely new, and never before seen in the world, the concept of digital scarcity enables the hard-capping of the bitcoin supply at 21 million, and acts as a cornerstone upon which the foundation of this new cypherpunk financial revolution is built.

“It is worth pointing out that Satoshi didn’t manage to make infor­ma­tion non-copyable. Every part of Bitcoin — its source code, the ledger, your private key — can be copied. All of it can be dupli­cated and tampered with. However, Satoshi managed to build a system that makes rule-breaking copies completely and utterly useless. The Bitcoin network performs an intri­cate dance to decide which copies are useful and which aren’t, and it is this dance that brings scarcity into the digital realm. And like with every dance, a temporal measuring stick is required to dictate the rhythm.”

GigiBitcoin is Time
  • Durable: Bitcoin cannot rust, it cannot age or degrade over time. It is durable across time and space.
  • Fungible: One of the core properties of money, bitcoin is extremely fungible
  • Verifiable: Bitcoin excels over gold, your full node instantly verifies if your bitcoin is authentic, no need for chemical assay tests, COAs, etc. You can verify for yourself, with no third party.
  • Portable and Divisible: Your money must be able to move, and you must be able to divide it. (You want to be able to get change when you go to the store…)
  • Unforgeable Costliness: A concept from Nick Szabo outlined in “Shelling Out, The Origins of Money.” There’s no way to “get free bitcoin” without someone having to work for it. The cost of extracting bitcoin from the network cannot be “faked.”

I’ve spent a lot of time on this layer, and it’s evolved over time. In versions one through three of the “Full Node Sculpture,” the Merkle trees looked like this, and my buddy, Johnny Dollar, gave me smack over the fact that there are no legitimate “Merkle trees” in the layer. I smiled and told him I can take artistic license… but his comments stuck with me.

Image for post

I felt I could do better here and I spent a long time re-imagining the tree design on this layer. After a lot of hard work, I came up with this. But still, I didn’t have “Merkle trees” really visualized in a way that Johnny would approve…

Image for post

Unfortunately, this design was too intricate for the laser to cut, so I had to scrap it and go back to scratch. Every time I tried to cut this, it would just burn the wood, and all details were lost. So, back to the drawing board. Literally.

After some more design work, I ended up with a design that both embodied “Merkle trees” but also hinted at the form of a mushroom, with mycelial networked threads connecting to peers, best described by Brandon Quittem in ”Bitcoin Is The Mycelium of Money.”

Not only that, but the laser did a pretty good job with the modified design. Just like Bitcoin, we must work within the constraints of the protocol.

Image for post

Layer 6: Block Height By Year

This ring has 132 sections that mirror the 132 years from 2009 to 2140, but instead of displaying the year, these containers display the block height at the end of the year.

Image for post
Here you can see rings six and seven combining to detail the block reward decrease at block height 630,000 from 12.5 to 6.25 BTC per block.

Layer 7: Block Reward By Halving Era

This ring is divided into 34 “Halving Eras” and tracks the bitcoin block subsidy issued by the network in four-year blocks over 132 years. Every time a new block is mined, new bitcoin are issued, and the amount geometrically decreases over time. We begin at 50 BTC per block in the first Halving Era from 2009 to 2012, and end in 2140 when the last bitcoin is mined, and the block reward drops to zero.

Layer 8: More Encoded Information

Image for post

After reading the below tweet from Saifedean Ammous, referencing the work by Plan B, I decided to add the stock-to-flow equations from Plan B’s GitHub into the node sculpture.

Layer 9: Bitcoin Per Day/Bitcoin Per Year

Image for post
Just like it says, this ring tracks the bitcoin issuance per day (top), and per year (bottom).

Above we can see that in 2009, the network produces 7,200 BTC per day, and 2,628,000 BTC per year in the first four-year Halving cycle. In our current era (pictured below), 2020 to 2024, the network produces only 900 BTC per day, which comes out to 328,500 per year.

Image for post

Layer 10: Bitcoin Total Issuance

This ring displays the Bitcoin total issuance at the end of each Halving cycle. Below we can see the first and last totals. At the end of the first Halving cycle, 10,500,000 BTC, or half of the total issuance has been mined. By the 34th era, all 21 million BTC will have been mined (there’s nuance here, as some coins have been destroyed/lost)

Image for post

Layer 11: The First Bitcoin Transaction

On January 11, 2009, in block 170, Satoshi sent Hal Finney 10 BTC in the first ever bitcoin transaction. That transaction is commemorated here in Layer 11. The ring details the nature of bitcoin spends to create “change addresses” in addition to “receive addresses.” To send Hal 10 BTC, Satoshi had to break up a $50 bill (of sorts, it’s a 50 BTC UTXO). To send Finney 10 BTC, Satoshi also had to send himself back 40 BTC. Which you can see here:

Image for post

Finney eventually spent the 10 BTC in this transaction in November 2011. Satoshi appears to have sent someone else (or maybe Finney again) another 10 BTC in this transaction the next day in block 181, but those 10 BTC are still unspent to this day.

Layer 12: The Halving Ruler

This ruler tracks every 2,016 blocks between each difficulty adjustment, and every 210,000 blocks between Halvings.

Image for post
  • Calendar Years: 132 (green arrow)
  • Difficulty Adjustments: 3,437 (blue arrow)
  • Reward Eras: 34 (purple arrow)
  • Block Height: 7,000,000 (red arrow)

Layer 13: Satoshi’s Equations

This ring contains all of the mathematical equations found in the Bitcoin white paper.

Image for post

Layer 14: Calendar

This ring is very simple, it tracks regular calendar time, over 365.4 days/year. It tracks month and day for use in linked timestamping through Merkle trees.

Image for post

Layer 15: Block Confirmation Time

In this layer we look at the Poisson distribution of block confirmation times. Some blocks take more time and some take less, but the network strives for an equilibrium at 10 minutes per block. This wavy distribution of time is captured in the 10 ARCs within the upper semicircle.

Inside the main circle with the Bitcoin logo, we have “hard coded” the genesis block into the full node, just like in every bitcoin full node:

  • scriptPubKey=0x5F1DF16B2B704C8A578D0B)
  • Merkle Root = 3BA3EDFD7A7B12B27AC72C3E67768F617FC81BC3888A51323A9FB8AA4B1E5E4A
  • Block 0 Transaction Hash = 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
  • Satoshi’s First Address — 1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa — a true Bitcoin mystery. For some reason, even though I believe no one can ever spend bitcoin from this address, people have been sending BTC to this address almost daily for 12-plus years. Why?
Image for post

Layers 16 to 19: Block Reward Eras

Image for post
Layers 16 to 19

Here we can clearly see the bitcoin block reward eras divided into;

  • Three Double-Digit Reward Eras (50, 25 and 12.5 BTC per Block)
  • Three Single-Digit Reward Eras (6.25, 3.125, 1.56)
  • 28 Sub-Full Bitcoin Reward Eras (less than 1 BTC per block)

Layer 20: Bitcoin Issuance Ruler

Tracks bitcoin issuance (from zero at the center) up to 21 million over time.

Image for post
Layers 20 and 21

Layer 21: Bitcoin Halving Era Ruler

Tracks the Halving reward eras in which bitcoin is issued (from zero to 34, even though 64 Halvings are programmed into the source code). Halvings 35 to 64 are set to zero bitcoin per block, so are not tracked in this ruler.

The Back: Bitcoin Signed COA (Certificate of Authenticity)

Most art comes with a certificate of authenticity, but these fragile paper certificates are simple to counterfeit, and only give the buyer a minimal assurance of the provenance of the artwork.

I was able to sign a message on-chain, and hide it inside a CoinJoin transaction.

This message, the COA itself, is stored immutably on the Bitcoin blockchain forever, and is a unique re-imagining of a concept from the traditional art world. We enter a meta-realm where the sculpture exists only because of Bitcoin, and now the sculpture is connecting back to the source in a never-ending circle, as the transaction is reconfirmed every 10 minutes.

Image for post
The hidden message on the Bitcoin blockchain.
Image for post
The transaction is called a “CoinJoin” as it is difficult to disambiguate inputs and outputs.

Here’s the transaction hash: 32dab765d6b5ffbc2bcd4e14adfb261e040b40be5f084415ce06a3ba9605a8f6

You can see my “creator address” — 1KZ8q6HKdTc1DsENqu722sDkrALGpdFXLi — below in the signed message, you can also see it as an input in the above transaction. You’ll notice I had to use a “legacy address” beginning with a 1. This is because Bitcoin Core can only sign messages from legacy and not native SegWit addresses. I thought I had created a cool address with “NGU” in it, but it was late, I was tricked, it’s a “q” not a “g.”

Image for post

And this is all engraved on the backside of the sculpture:

Image for post
Image for post
Image for post
Image for post

A Few More Pictures For The Road…

Image for post
Image for post

Functionality?

So, okay, right now, the sculpture doesn’t really “do” anything other than look pretty hanging on the wall. However there’s a lot planned and some of those plans are described below.

Section 3: The Future

We have a long way to go, in some ways, we’re just beginning. I have visions of fully-mechanized moving parts, interlocking rings and embedded small board computers running Bitcoin Core within the sculpture. Now that we have the map, we have to embark on the journey. All it takes is moving the clock forward.

I’m currently working on a project that will allow live Bitcoin network data to be light mapped onto the full node sculpture (using projection mapping techniques). A friend and I did a proof of concept project at last year’s Bitcoin Hackathon and you can see some photos from my presentation below:

Image for post
Image for post
Image for post

The video below is what is to be projected onto the full node and was an example of “finding a new block.” Keep in mind that the below was done over a 24-hour period, with almost no sleep, but I feel it’s still a good representation of the idea here. With more time and focus, I’m excited to see where we can go with this!

I believe bitcoin has a lot to teach us about being human, interacting with others and our environment, and growing as a species.

I leave you with these thoughts…

Bitcoin is crystallized time and energy. In this form, they can be stored indefinitely and transported instantly across the planet. Bitcoin can exist in the physical world, the digital world and it can even be stored in your mind, as nothing more than a memory. We are living in amazing times.

“Timekeeping devices have trans­formed civiliza­tions more than once. As Lewis Mumford pointed out in 1934: ‘The clock, not the steam-engine, is the key-machine of the modern indus­trial age.’ Today, it is again a timekeeping device that is trans­forming our civiliza­tion: a clock, not computers, is the true key-machine of the modern infor­ma­tional age. And this clock is Bitcoin.”

--Gigi, “Bitcoin Is Time”

This is a guest post by FractalEncrypt. Opinions expressed are entirely their own and do not necessarily reflect those of BTC Inc or Bitcoin Magazine.

The post Decoding The “Bitcoin Full Node Sculpture,” A Cypherpunk Chronometer appeared first on Bitcoin Magazine.

Read the whole story
kleer001
176 days ago
reply
Oh wow, that's really beautiful. 😭🖖
Share this story
Delete

mayflower-gal:serenata-your-neighborhood-lefty: “I am, somehow, less interested in the weight...

2 Comments and 14 Shares

mayflower-gal:

serenata-your-neighborhood-lefty:

image

“I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops.”

-Stephen Jay Gould.

Read the whole story
kleer001
178 days ago
reply
My complaint is that the centralization of Socialism makes disasters of corruption inevitable. Democratic socialism however is awesome.
popular
197 days ago
reply
Share this story
Delete
1 public comment
sness
195 days ago
reply
i agree
milky way

Saturday Morning Breakfast Cereal - Science Fictions

2 Comments and 12 Shares


Click here to go see the bonus panel!

Hovertext:
If anyone's having deja vu, this was run in Nautilus magazine a while back. I'm linking to the amazon page where you can buy.


Today's News:
Read the whole story
kleer001
179 days ago
reply
Say it again for the kids in the back
Share this story
Delete
1 public comment
jlvanderzwan
180 days ago
reply
Wasn't this already published before?
Levitz
170 days ago
Hovertext helps: "If anyone's having deja vu, this was run in Nautilus magazine a while back." So yes :)

What Has 2020 Taught You?

1 Comment

this is a diesel sweeties comic strip

This is technically a remix of a 2016 comic, but we’ve been in a time loop, so it’s OK.

Read the whole story
kleer001
206 days ago
reply
Gripping your pillow tight
Share this story
Delete

MediaPipe Holistic — Simultaneous Face, Hand and Pose Prediction, on Device

1 Comment and 2 Shares

Real-time, simultaneous perception of human pose, face landmarks and hand tracking on mobile devices can enable a variety of impactful applications, such as fitness and sport analysis, gesture control and sign language recognition, augmented reality effects and more. MediaPipe, an open-source framework designed specifically for complex perception pipelines leveraging accelerated inference (e.g., GPU or CPU), already offers fast and accurate, yet separate, solutions for these tasks. Combining them all in real-time into a semantically consistent end-to-end solution is a uniquely difficult problem requiring simultaneous inference of multiple, dependent neural networks.

Today, we are excited to announce MediaPipe Holistic, a solution to this challenge that provides a novel state-of-the-art human pose topology that unlocks novel use cases. MediaPipe Holistic consists of a new pipeline with optimized pose, face and hand components that each run in real-time, with minimum memory transfer between their inference backends, and added support for interchangeability of the three components, depending on the quality/speed tradeoffs. When including all three components, MediaPipe Holistic provides a unified topology for a groundbreaking 540+ keypoints (33 pose, 21 per-hand and 468 facial landmarks) and achieves near real-time performance on mobile devices. MediaPipe Holistic is being released as part of MediaPipe and is available on-device for mobile (Android, iOS) and desktop. We are also introducing MediaPipe’s new ready-to-use APIs for research (Python) and web (JavaScript) to ease access to the technology.

Top: MediaPipe Holistic results on sport and dance use-cases. Bottom: “Silence” and “Hello” gestures. Note, that our solution consistently identifies a hand as either right (blue color) or left (orange color).

Pipeline and Quality
The MediaPipe Holistic pipeline integrates separate models for pose, face and hand components, each of which are optimized for their particular domain. However, because of their different specializations, the input to one component is not well-suited for the others. The pose estimation model, for example, takes a lower, fixed resolution video frame (256x256) as input. But if one were to crop the hand and face regions from that image to pass to their respective models, the image resolution would be too low for accurate articulation. Therefore, we designed MediaPipe Holistic as a multi-stage pipeline, which treats the different regions using a region appropriate image resolution.

First, MediaPipe Holistic estimates the human pose with BlazePose’s pose detector and subsequent keypoint model. Then, using the inferred pose key points, it derives three regions of interest (ROI) crops for each hand (2x) and the face, and employs a re-crop model to improve the ROI (details below). The pipeline then crops the full-resolution input frame to these ROIs and applies task-specific face and hand models to estimate their corresponding keypoints. Finally, all key points are merged with those of the pose model to yield the full 540+ keypoints.

MediaPipe Holistic pipeline overview.

To streamline the identification of ROIs, a tracking approach similar to the one used for the standalone face and hand pipelines is utilized. This approach assumes that the object doesn't move significantly between frames, using an estimation from the previous frame as a guide to the object region in the current one. However, during fast movements, the tracker can lose the target, which requires the detector to re-localize it in the image. MediaPipe Holistic uses pose prediction (on every frame) as an additional ROI prior to reduce the response time of the pipeline when reacting to fast movements. This also enables the model to retain semantic consistency across the body and its parts by preventing a mixup between left and right hands or body parts of one person in the frame with another.

In addition, the resolution of the input frame to the pose model is low enough that the resulting ROIs for face and hands are still too inaccurate to guide the re-cropping of those regions, which require a precise input crop to remain lightweight. To close this accuracy gap we use lightweight face and hand re-crop models that play the role of spatial transformers and cost only ~10% of the corresponding model's inference time.

 MEH   FLE 
 Tracking pipeline (baseline)   9.8%   3.1% 
 Pipeline without re-crops   11.8%   3.5% 
 Pipeline with re-crops   9.7%   3.1% 
Hand prediction quality.The mean error per hand (MEH) is normalized by the hand size. The face landmarks error (FLE) is normalized by the inter-pupillary distance.

Performance
MediaPipe Holistic requires coordination between up to 8 models per frame — 1 pose detector, 1 pose landmark model, 3 re-crop models and 3 keypoint models for hands and face. While building this solution, we optimized not only machine learning models, but also pre- and post-processing algorithms (e.g., affine transformations), which take significant time on most devices due to pipeline complexity. In this case, moving all the pre-processing computations to GPU resulted in ~1.5 times overall pipeline speedup depending on the device. As a result, MediaPipe Holistic runs in near real-time performance even on mid-tier devices and in the browser.

 Phone   FPS 
 Google Pixel 2 XL   18 
 Samsung S9+   20 
 15-inch MacBook Pro 2017   15 
Performance on various mid-tier devices, measured in frames per second (FPS) using TFLite GPU.

The multi-stage nature of the pipeline provides two more performance benefits. As models are mostly independent, they can be replaced with lighter or heavier versions (or turned off completely) depending on the performance and accuracy requirements. Also, once pose is inferred, one knows precisely whether hands and face are within the frame bounds, allowing the pipeline to skip inference on those body parts.

Applications
MediaPipe Holistic, with its 540+ key points, aims to enable a holistic, simultaneous perception of body language, gesture and facial expressions. Its blended approach enables remote gesture interfaces, as well as full-body AR, sports analytics, and sign language recognition. To demonstrate the quality and performance of the MediaPipe Holistic, we built a simple remote control interface that runs locally in the browser and enables a compelling user interaction, no mouse or keyboard required. The user can manipulate objects on the screen, type on a virtual keyboard while sitting on the sofa, and point to or touch specific face regions (e.g., mute or turn off the camera). Underneath it relies on accurate hand detection with subsequent gesture recognition mapped to a "trackpad" space anchored to the user’s shoulder, enabling remote control from up to 4 meters.

This technique for gesture control can unlock various novel use-cases when other human-computer interaction modalities are not convenient. Try it out in our web demo and prototype your own ideas with it.

In-browser touchless control demos. Left: Palm picker, touch interface, keyboard. Right: Distant touchless keyboard. Try it out!

MediaPipe for Research and Web
To accelerate ML research as well as its adoption in the web developer community, MediaPipe now offers ready-to-use, yet customizable ML solutions in Python and in JavaScript. We are starting with those in our previous publications: Face Mesh, Hands and Pose, including MediaPipe Holistic, with many more to come. Try them directly in the web browser: for Python using the notebooks in MediaPipe on Google Colab, and for JavaScript with your own webcam input in MediaPipe on CodePen!

Conclusion
We hope the release of MediaPipe Holistic will inspire the research and development community members to build new unique applications. We anticipate that these pipelines will open up avenues for future research into challenging domains, such as sign-language recognition, touchless control interfaces, or other complex use cases. We are looking forward to seeing what you can build with it!

Complex and dynamic hand gestures. Videos by Dr. Bill Vicars, used with permission.

Acknowledgments
Special thanks to all our team members who worked on the tech with us: Fan Zhang, Gregory Karpiak, Kanstantsin Sokal, Juhyun Lee, Hadon Nash, Chuo-Ling Chang, Jiuqiang Tang, Nikolay Chirkov, Camillo Lugaresi, George Sung, Michael Hays, Tyler Mullen, Chris McClanahan, Ekaterina Ignasheva, Marat Dukhan, Artsiom Ablavatski, Yury Kartynnik, Karthik Raveendran, Andrei Vakunov, Andrei Tkachenka, Suril Shah, Buck Bourdon, Ming Guang Yong, Esha Uboweja, Siarhei Kazakou, Andrei Kulik, Matsvei Zhdanovich, and Matthias Grundmann.

Read the whole story
kleer001
219 days ago
reply
Share this story
Delete
1 public comment
sness
224 days ago
reply
there you go
milky way
Next Page of Stories