Contents^
Items^
Bitcoin Is Time
I'm ready to be downvoted to oblivion, especially by all the Bitcoin purists, but I hope my message sparks at least some curiosity.
Bitcoin is getting "weaker" every year for several reasons:
- emerging centralization due to economies of scale. If this trend continues the mining power will be so consolidated that a 51% attack will be likely. The Nakamoto coefficient is already at 4, and tending towards 3.
- energy usage due to Proof of Work is growing astronomically, and the higher the price of bitcoin, the less incentive there is to use renewable energy. Bitcoin uses more energy than the country of Argentina.
- transaction times are SLOW, and expensive. The lightning network is incredibly buggy and won't actually solve the problems it's promising.
There is a better alternative. RaiBlocks, named nano since 2018. It got a bad rap due to the BitGrail hack, but it's picking up steam again. The developer community is great, the main dev team on the Nano protocol has been consistently chugging along regardless of the 3 years of crypto winter on a shoestring budget and the nano community is made up of users, not speculators.
This article sums it up quite nicely: https://senatusspqr.medium.com/why-nano-is-the-ultimate-stor...
I'm happy to receive downvotes, but all I ask in return is that you give it a read with an open mind.
> The lightning network is incredibly buggy and won't actually solve the problems it's promising.
Not an expert but I found that a readable summary of different layer 2 scaling strategies and why Ethereum developers prefer Rollups instead of Channels (like Lightning):
"Bitcoin scalability problem" could link to the Ethereum design docs: https://en.wikipedia.org/wiki/Bitcoin_scalability_problem
The Ethereum design docs could link to direct-listed premined [stable] coins as a solution for Proof of Work and TPS reports: https://github.com/flare-eng/coston#smart-contracts-with-xrp
(edit) re: n-layer solutions: The https://interledger.org/ RFCs and something like Transaction Permission Layer (TPL) will probably be helpful for interchain compliance.
> Interledger is not tied to a single company, blockchain, or currency.
From https://tplprotocol.org/ :
> The challenge: Current blockchain-based protocols lack an effective governance mechanism that ensures token transfers comply with requirements set by the project that issued the token.
> Projects need to set requirements for a variety of reasons. For instance, remaining compliant with securities laws, limiting transfer to beta testers, or limiting transfer to a particular geo-spatial location. Whatever your reason, if a requirement can be verified by a third-party, TPL will be able to help.
In the US, S-Corps can't have international or more than n shareholders, for example; so if firms even wanted to issue securities on a first-layer network, they'd need an extra-chain compliance mechanism to ensure that their issuance is legal pursuant to local, sovereign, necessary policies. Re-issuing stock certificates is something that has to be done sometimes. When is it possible to cancel outstanding tokens?
Foundational Distributed Systems Papers
From "Ask HN: Learning about distributed systems?" https://news.ycombinator.com/item?id=23932271 :
> Papers-we-love > Distributed Systems: https://github.com/papers-we-love/papers-we-love/tree/master...
> awesome-distributed-systems also has many links to theory: https://github.com/theanalyst/awesome-distributed-systems
And links to more lists of distributed systems papers under "Meta Lists": https://github.com/theanalyst/awesome-distributed-systems#me...
In reviewing this awesome list, today I learned about this playlist: "MIT 6.824 Distributed Systems (Spring 2020)" https://youtube.com/playlist?list=PLrw6a1wE39_tb2fErI4-WkMbs...
> awesome-bigdata lists a number of tools: https://github.com/onurakpolat/awesome-bigdata
Low-Cost Multi-touch Whiteboard using the Wiimote (2007) [video]
"Interactive whiteboard" / "smart board" https://en.wikipedia.org/wiki/Interactive_whiteboard
Wii Remote > Features > Sensing: https://en.wikipedia.org/wiki/Wii_Remote#Sensing
.. > Third-Party Development describes a number of applications for IR/optical tracking with an array of nonstationary emitters: https://en.wikipedia.org/wiki/Wii_Remote#Third-party_develop...
Augmented Reality (AR) > Technology > Tracking: https://en.wikipedia.org/wiki/Augmented_reality#Tracking
... links to "VR positional tracking" which does have headings for "Optical" and "Sensor fusion": https://en.wikipedia.org/wiki/VR_positional_tracking
How to Efficiently Choose the Right Database for Your Applications
I kinda disagree with separate branch for "document database" for Mongo. Mongo is a key-value storage, with a thin wrapper that converts BSON<->JSON, and indices on subfields.
You can achieve exactly the same thing with PostgreSQL tables with two columns (key JSONB PRIMARY KEY, value JSONB), including indices on subfields. With way more other functionality and support options.
> You can achieve exactly the same thing with PostgreSQL tables with two columns (key JSONB PRIMARY KEY, value JSONB), including indices on subfields. With way more other functionality and support options.
PostgreSQL docs > "JSON Functions and Operators" https://www.postgresql.org/docs/current/functions-json.html
MongoDB can do jsonSchema:
> Document Validator¶ You can use $jsonSchema in a document validator to enforce the specified schema on insert and update operations:
db.createCollection( <collection>, { validator: { $jsonSchema: <schema> } } )
db.runCommand( { collMod: <collection>, validator:{ $jsonSchema: <schema> } } )
Looks like there are at least 2 ways to handle JSONschema with Postgres: https://stackoverflow.com/questions/22228525/json-schema-val... ; neither of which are written in e.g. Rust or Go.
Is there a good way to handle JSON-LD (JSON Linked Data) with Postgres yet?
There are probably 10 comparisons of triple stores with rule inference slash reasoning on data ingress and/or egress.
A Data Pipeline Is a Materialized View
Sometime when I am old(er) and (somehow?) have more time, I'd like to jot down a "Rosetta Stone" of which buzzwords map to the same concepts. So often we change our vocabulary every decade without changing what we're really talking about.
Things started out in a scholarly vein, but the rush of commerce hasn't allowed much time to think where we're going. — James Thornton, Considerations in computer design (1963)
Like a Linked Data thesaurus with typed, reified edges between nodes/concepts/class_instances?
Here's the WordNet RDF Linked Data for "jargon"; like the "Jargon File": http://wordnet-rdf.princeton.edu/lemma/jargon
A Semantic MediaWiki Thesaurus? https://en.wikipedia.org/wiki/Semantic_MediaWiki :
> Semantic MediaWiki (SMW) is an extension to MediaWiki that allows for annotating semantic data within wiki pages, thus turning a wiki that incorporates the extension into a semantic wiki. Data that has been encoded can be used in semantic searches, used for aggregation of pages, displayed in formats like maps, calendars and graphs, and exported to the outside world via formats like RDF and CSV.
Google Books NGram viewer has "word phrase" term occurrence data by year, from books: https://books.google.com/ngrams
There’s no such thing as “a startup within a big company”
When I was at PowerBI in Microsoft, all the execs hailed it as Startup within Microsoft. Come work here instead of Uber. I worked like a dog, sometimes till 2am in morning. My manager would routinely ask us to come on weekends. I was naive, I thought we are growing customer base, this is what a startup looks like.
The ultimate realization was in a startup you have equity, a decent amount in a good startup. At Microsoft it was a base salary and set amount of stock. What we did moved very little of the top revenue metric. It made little difference if I worked like a dog, or slacked. The promos were very much “buddy buddy” system.
In the end I realized you can’t have startups in big companies (esp as an engineer you don’t have the huge upside if the startup is successful, your upside is capped)
Startups work because you have skin in the game, when you build something people want, you get to reap rewards proportional to it. That correlation and feedback loop is very important.
At big companies you don’t have the the same correlation. Some big shot exec they hired reaps far far more on the work you did.
Equity is what builds wealth.
>The ultimate realization was in a startup you have equity, a decent amount in a good startup. At Microsoft it was a base salary and set amount of stock.
I joined a startup in 1999. There were 3 founders and I was employee #2 after that. I received a ton of options (this was before RSUs became popular). We had a great product and a great team, but 18 months later ran out of money and unfortunately it was right after the dotcom implosion of early 2001 when funding had completely dried up.
My takeaway was that base salary is actually the most important component of TC. Cash bonus based on some metric that you control comes second. Equity comes third. If you work for a FAANG, maybe equity can move higher up (though it remains to be seen how long this will be true).
Outside of FAANG (and top executives at F500 sized public companies) very few people are getting rich off of the "equity" component of their TC. The vast majority of startups go bust before IPO or acquisition.
Time is the most valuable commodity you have, don't squander it for lottery tickets and empty promises.
>Time is the most valuable commodity you have, don't squander it for lottery tickets and empty promises.
That goes the other way around too, the only way to have massive amounts of free time without going FIRE is to win the lottery. So why not throw the dice once or twice when you're young and then settle into a stable corporate job if it doesn't pay out?
Modern Silicon Valley lets you do both. Why not throw the dice and still make around $200k cash?
As a [good] engineer you can have your cake and eat it too. Sure it won’t make you a billionaire but you can live a comfy life, maybe win the lottery, and retire at 50 if the lottery fails.
Living and working elsewhere with the wages of the region reduces expenses and opportunities; but the wealth of educational resources online [1][2] does make it feasible to even bootstrap a company on the side. Do you need to borrow money to scale quickly enough to pay expenses with sufficient cash flow for the foreseeable future?
Income sources: Passive income, Content, Equity that's potentially worth nothing, a backtested diversified portfolio (Golden Butterfly or All Weather Portfolio and why?) of sustainable investments, Business models [3]; Software implementations of solutions to businesses, organizations, and/or consumers' opportunities
Single-payer / Universal Healthcare is a looming family expense for many entrepreneurs; many of whom do get into entrepreneurship later in life.
Small businesses make up a significant portion of GDP. Small businesses have to have to accept risk.
There's still opportunity in the world.
[1] Startup School > Curriculum https://www.startupschool.org/curriculum
[2] https://www.ycombinator.com/library
[3] "Business models based on the compiled list at [HN]" https://gist.github.com/ndarville/4295324
From "Why companies lose their best innovators (2019)" https://news.ycombinator.com/item?id=23887903 :
> "Intrapreneurial." What does that even mean? The employee, within their specialized department, spends resources (time, money, equipment) on something that their superior managers have not allocated funding for because they want: (a) recognition; (b) job security; (c) to save resources such as time and money; (d) to work on something else instead of this wasteful process; (e) more money.
I’d you are going to start something on the side, carefully read your current employment agreement, particularly the part about IP assignment. Most Silicon Valley companies I’ve worked with, including FAANGs, claim ownership of everything you produce, inside or outside of employment, at home or in office, using their equipment or yours. You don’t want to lick into a unicorn idea and have your former employer’s lawyers send you that letter...
Unusual data point (I'm not sure if South African practice is applicable elsewhere) , but most bigger corps I've worked at have provided a mechanism to declare external interests, pending management approval, to work around these global clauses.
On paper any IP would be default belong to the Corp, but after some discussion/negotiations with your line manager and his higher-ups, you could do the paper work to declare.
Are these actually enforceable?
It probably doesn’t matter in practice. They will have more lawyers and more money to burn on legal process than you. Who will go bankrupt first fighting a legal battle between you and, say, Apple?
In practice, I don't think I've ever seen a SV example of that kind of litigation from garage development of IP. Maybe I've just missed them?
Theft of IP from one company to a new company on the other hand there are multiple public examples of.
The only time they'd actually sue is if you made something worth money. in that case, there's a good chance you can get investors to pay the lawsuit.
It does matter in practice in California.
If its "related" to your employers business yes - unless you agree a variation of the contract.
Ask HN: Keyrings: per-package/repo; commit, merge, and release keyrings?
Are there existing specs for specifying per-package release keyrings and per-repo commit and merge keyrings?
Keyring: a collection of keys imported into a datastore with review.
DevOpsSec; Software Supply Chain Security
Packages {X, Y, Z} in Indexes {A, B, C} are artifacts that are output from Builds (on workstations or servers with security policies) which build a build script (which is often deliberately not specified with a complete programming language in order to minimize build complexity; instead preferring YAML) which should be drawn from a stable commit hash in a Repository (which may be a copy of technically zero or more branches of a Repository hosted centrally next to Issues and Build logs and Build artifact Signing Keys).
Maxmimally, are there potentially more keyrings (or key authorization mappings between key and permission) than (1) commit; (2) merge; and (3) release?
Source Projects: Commit, Merge, [Run Build, Login to post-build env], Release (and Sign) package
Downstream Distros: Commit, Merge, [Run Build, Login to post-build env], Release (and Sign) package for the {testing, stable, security} (Signed) Index catalogs
Threat Actors Now Target Docker via Container Escape Features
It may be quipped that Docker or Kubernetes is remote code execution (RCE) as a service!
The vulnerability pointed out here is server operators who have exposed Docker's API to the public internet, allowing anyone to run a container. The use of a privileged container is just icing on the cake, and probably not nessecary for a cryptominer.
A more subtle and interesting attack vector is genuine container imags that become compromised, much like any other package manager, particularly ones that search for the Docker API on the host's internal IP address, hoping only public network access is firewalled.
Docker engine docs > "Protect the Docker daemon socket" https://docs.docker.com/engine/security/protect-access/
dev-sec/cis-docker-benchmark /controls: https://github.com/dev-sec/cis-docker-benchmark/tree/master/...
Eh. This advice is less practical than it’s made to seem. Like it “works” but it’s not really usable for anything other than connecting two privileged apps over a hostile network.
* Docker doesn’t support CRLs so any compromised cert means reissuing everyone’s cert.
* Docker’s permissions are all or nothing without a plug-in. And if you’re going that route the plug-in probably has better authentication.
* Docker’s check is just “is the cert signed by the CA” so you have to do one CA per machine / group homogeneous machines.
* You either get access to the socket or not with no concept of users so you get zero auditing.
* Using SSH as transport helps but then you have to also lock down SSH which isn’t impossible but more work and surface area to cover than feels necessary. Also since your access is still via the Unix socket it’s all or non permissions again.*
django-ca is one way to manage a PKI including ACMEv2, OCSP, and a CRL (Certificate Revocation) list: https://github.com/mathiasertl/django-ca
"How can I verify client certificates against a CRL in Golang?" mentions a bit about crypto/tls and one position on CRLs: https://stackoverflow.com/questions/37058322/how-can-i-verif...
CT (Certificate Transparency) is another approach to validating certs wherein x.509 cert logs are written to a consistent, available blockchain (or in e.g. google/trillian, a centralized db where one party has root and backup responsibilities also with Merkle hashes for verifying data integrity). https://certificate.transparency.dev/ https://github.com/google/trillian
Does docker ever make the docker socket available over the network, over an un-firewalled port by default? Docker Swarm is one config where the docker socket is configured to be available over TLS.
Docker Swarm docs > "Manage swarm security with public key infrastructure (PKI)" https://docs.docker.com/engine/swarm/how-swarm-mode-works/pk... :
> Run `docker swarm ca --rotate` to generate a new CA certificate and key. If you prefer, you can pass the --ca-cert and --external-ca flags to specify the root certificate and to use a root CA external to the swarm. Alternately, you can pass the --ca-cert and --ca-key flags to specify the exact certificate and key you would like the swarm to use.
Docker ("moby") and podman v3 socket security could be improved:
> From "ENH,SEC: Create additional sockets with limited permissions" https://github.com/moby/moby/issues/38879 ::
> > An example use case: securing the Traefik docker driver:
> > - "Docker integration: Exposing Docker socket to Traefik container is a serious security risk" https://github.com/traefik/traefik/issues/4174#issuecomment-...
> > > It seems it only require (read) operations : ServerVersion, ContainerList, ContainerInspect, ServiceList, NetworkList, TaskList & Events.
> > - https://github.com/liquidat/ansible-role-traefik
> > > This role does exactly that: it launches two containers, a traefik one and another to securely provide limited access to the docker socket. It also provides the necessary configuration.
> > - ["What could docker do to make it easier to do this correctly?"] https://github.com/Tecnativa/docker-socket-proxy/issues/13
> > - [docker-socket-proxy] Creates a HAproxy container that proxies limited access to the [docker] socket
This looks like the attacker is just using a publicly exposed Docker API honeypot to run a new Docker container with `privileged: true`. I don't see why that's particularly interesting given that they could just bind-mount the host / filesystem with `pid: host` and do pretty much whatever they want?
It would be much more interesting to see a remote-code-execution attack against some vulnerable service deploying a container escape payload to escalate privileges from the Docker container to the host.
Exposed on the WAN is obviously bad, but how do you keep your own containers from calling those APIs? Yes, you don't mount the docker socket in the container, but what about the orchestrator APIs?
With kubernetes, you enable RBAC, and only allow each pod the absolute minimum of access required.
In my own setup, I have only five pods with any privileges: kubernetes-dashboard, nginx-ingress, prometheus, cert-manager, and the gitlab-runner, and of those only kubernetes-dashboard can modify existing pods in all spaces, cert-manager can only write secrets of its own custom type, and gitlab-runner can spawn non-privileged pods in its own namespace (but not mount anything from the host, nor any resources from any other namespace).
And when building docker images in the CI, I use google’s kaniko to build docker images from within docker without any privileges (it unpacks docker images for building and runs them inside the existing container, basically just chroot).
All these APIs are marked very clearly and obviously as DO NOT MAKE AVAILABLE PUBLICLY. If you still make them public, well it’s pretty much your own fault.
Does RBAC not limit these by default? Does cert-manger not already give itself restricted permission on install? Do I need to fix up my cluster right now? If so, do you have any example RBAC yamls? :D
> And when building docker images in the CI, I use google’s kaniko to build docker images from within docker without any privileges (it unpacks docker images for building and runs them inside the existing container, basically just chroot).
You can also use standalone buildkit which comes with the added benefit of being able to use the same builder locally natively.
No RBAC doesn't automatically do this. And many publicly available Helm charts are missing these basic security configurations. You should use Gatekeeper or similar to enforce these settings throughout your cluster.
Gatekeeper docs: https://open-policy-agent.github.io/gatekeeper/website/docs/
Gatekeeper src: https://github.com/open-policy-agent/gatekeeper
awesome-container-security: https://github.com/kai5263499/awesome-container-security
"container-security" GitHub label: https://github.com/topics/container-security
In somewhat related news, rootless Docker support is no longer an experimental feature. Has anyone used it and liked/disliked it?
I have, and it's great/you forget about it pretty quickly. I've actually gone (rootful?)docker->(rootless)podman->(rootless)docker, and I've written about it:
- https://vadosware.io/post/rootless-containers-in-2020-on-arc...
- https://vadosware.io/post/bits-and-bobs-with-podman/
- https://vadosware.io/post/back-to-docker-after-issues-with-p...
I still somewhat long after podman's lack of a daemon but some of the other issues with it currently leave it completely out for me (now that docker has caught up) -- for example lack of linking support.
Podman works great outside of those issues though, you just have to be careful if it does something in a way that expects docker semantics (usually the daemon or an unix socket on disk somewhere). You'll also find some cutting edge tools like dive[0] also have some support for podman[1] but it's not necessarily the usual case.
podman v3 has a docker-compose compatible socket. From https://news.ycombinator.com/item?id=26107022 :
> "Using Podman and Docker Compose" https://podman.io/blogs/2021/01/11/podman-compose.html
Ask HN: What security is in place for bank-to-bank EFT?
When I set up an ETF on a bank's website, all I need to enter is the other bank's routing number and account number, which can be readily found on a paper check. Then you can transfer money from one bank to another... What security and authentication is in place to prevent fraud? In case of fraud, is the victim guaranteed to get the money back?
AFAIU, no existing banking transaction systems require the receiver to confirm in order to receive a funds transfer.
You can create a "multisig" DLT smart contract that requires multiple parties' signatures before the [optionally escrowed] funds are actually transferred.
EFT: Electronic Funds Transfer: https://en.wikipedia.org/wiki/Electronic_funds_transfer
As far as permissions to write to the account ledger: Check signatures are scanned. Cryptoasset keys are very long, high-entropy "passwords". US debit cards are chip+pin; it's not enough to just copy down the card number (and CVV code).
Though credit cards typically are covered by fraud protection, debit card transactions typically aren't: hopefully something will be recovered, but AFAIU debit txs might as well be as unreversible as cryptoasset transactions.
TPL: Transaction Permission Layer is one proposed system for permissions in blockchain; so that e.g. {proof of residence, receiver confirmation, accredited investor status, etc.} can be necessary for a transaction to go through.
ILP: Interledger Protocol > RFC 32 > "Peering, Clearing and Settling" describes how ~EFT with Interledger works: https://interledger.org/rfcs/0032-peering-clearing-settlemen...
Podman: A Daemonless Container Engine
Is the title of this page out of date?
AFAIU, Podman v3 has a docker-compose compatible socket and there's a daemon; so "Daemonless Container Engine" is no longer accurate.
"Using Podman and Docker Compose" https://podman.io/blogs/2021/01/11/podman-compose.html
Podman does not run in a daemon-mode by default. There's a dbus-activated service unit that activates the daemon when you try to use Docker Compose with it.
It is almost 100% compatible with docker. I only had to do minimal changes in my Dockerfiles to use it.
But not Buildkit and docker-compose.
Podman v3 is compatible with docker-compose (but not yet swarm mode, FWIU), has a socket and a daemon that services it.
Buildah (`podman buildx`, `buildah bud --arch arm64`) just gained multiarch build support; so also building arm64 containers from the same Dockerfile is easy now. https://github.com/containers/buildah/issues/1590
IDK what BuildKit features should be added to Buildah, too?
> IDK what BuildKit features should be added to Buildah, too?
Cache mounts are what we rely on for incremental rebuilds.
Cambridge Bitcoin Electricity Consumption Index
Ok, I have to admit that I am one of those people who knew about BTC very, very early but never bought any. Now, of course I should have mined at least 100 back in the days, but anyways.
The reason I have never mined nor bought into it is that I still, to this day struggle to come up with a real reason to do so.
Am I getting too old? Do I not see the "huge potential" what it may become? Don't I want to get freaking rich? I simply can't answer what BTCs and other crypto coins are actually good for. Can someone help me out here?
I'm in your boat, when it was first explained to me in a hacker space at college when it was first growing and priced at a few cents per 100++ Bitcoin. All I could see was a waste of energy and resources (time, energy, hardware).
It will always be a drain on society, and it scares me that it hasn't failed because we see it draining our GPU industry to the point that market prices have gone insane and availability is practically non-existent / all left up to chance.
It does take a rocket science to see the trends, as long as it's tied to needing hardware to compute arbitrary calculations to work... It's going to waste computer resources and energy. Both which are finite resources... Yes finite, solar cells aren't free to make, they don't last forever after you make them. Same with any other green technology. Maybe if fusion was our only power source it would be less of a concern, but we still haven't cracked Q=1, so... Yes it is a waste of finite resources.
It’s not draining the GPU industry. No one buys GPUs for mining Bitcoin anymore, they use ASICs.
Even as just ASICs there's something of a drain, Bitcoin ASICs consume fab production which means less remaining production capacity for GPUs.
This is an equilibrium that should eventually fix itself, but while Bitcoin and crypto mining is growing they are making a material dent in the global semiconductor economy.
Cryptoasset mining creates demand for custom chip fab (how different are mining rigs from SSL/TLS accelerator expansion cards), which is definitely not zero sum: more revenue = more opportunities.
https://en.wikipedia.org/wiki/Price_elasticity_of_supply
With insufficient demand, a market does not develop into a sustainable market. "Rule of three (economics)" says that markets are stable with 3 major competitors and many smaller competitors; nonlinearity and game theory.
https://en.wikipedia.org/wiki/Rule_of_three_(economics)
We've always had custom chip fab, but the prices used to be much higher. Proof of Work (and Proof of Research) incentivize microchip and software energy efficiency; whereas we had observed and been most concerned with doublings in transistor density.
FWIU, it's now more sustainable and profitable to mine rare earth elements from recycled electronics than actually digging real value out of the earth?
Compared to creating real value by digging for gold, how do we value financial services?
Bitcoin's fundamental value is negative given its environmental impact
If the price of energy is calculated with corresponding carbon tax included, shouldn't Bitcoin be neutral?
> If the price of energy is calculated with corresponding carbon tax included, shouldn't Bitcoin be neutral?
Yes, but there are vastly more energy efficient substitute DLTs with near-zero switching costs. Litecoin and scrypt (instead of AES256), for example.
Apply a USD/kWhr threshold across all industries.
Is this change (and focus on the external costs of energy production) more the result of penalties or incentives?
Pre-mined coins are vastly more energy efficient (with tx costs <1¢ and similarly minimal kWhr/tx costs), but the market doesn't trust undefined escrow terms that are fair game in commodities and retail markets.
We have trouble otherwise storing energy from noon to commute and dinner time; whereas a commodity like grain may keep for quite awhile.
Bitcoin serves as a demand subsidy when heavily-subsidized energy prices crash due to oversupply (that we should recognize as temporary because we are moving to electric vehicles and we need to reach production volumes so that, in comparison to alternatives, renewables are now more cost effective)
In the US, we have neither carbon taxes nor intraday prices. The EU has carbon taxes and electrical energy markets.
>In the US, we have neither carbon taxes nor intraday prices.
Like, no intraday electricity market?
Ask HN: What are some books where the reader learns by building projects?
2021 Edition. This is a continuation of the previous two threads which can be found here:
https://news.ycombinator.com/item?id=22299180
https://news.ycombinator.com/item?id=13660086
Other resources:
https://github.com/danistefanovic/build-your-own-x
https://github.com/AlgoryL/Projects-from-Scratch
https://github.com/tuvtran/project-based-learning
"Agile Web Development with Rails [6]" (2020) teaches TDD and agile in conjunction with a DRY, CoC, RAD web application framework: https://g.co/kgs/GNqnWV
Is it wrong to demand features in open-source projects?
Yes, it's wrong to demand something for nothing: that's entitlement, not business (which involves some sort of equitable exchange of goods and/or services).
Better questions: How do I file a BUG report issue, create a feature ENHancement request issue, send a pull request with a typo fix, write DOCs and send a PR, write test cases for a bug report?
How can I sponsor development of a feature?
A project may define a `.github/FUNDING.yml`, which GitHub will display on the 'Sponsor' tab of the GitHub project. A project may also or instead include funding information in their /README.md.
How do I ask IRC or a mailing list or issues how much and how long it would cost to develop a feature, if somebody had some international stablecoin and a limited term agreement?
The answer may be something like, "thanks for the detailed use case or user story, that's on our roadmap, there are major issues blocking similar features and that's where the expense would be."
CompilerGym: A toolkit for reinforcement learning for compiler optimization
What a great idea. Would per-instruction/opcode costs make this easier to optimize?
Are MuZero or the OpenAI baselines useful for RL compiler optimization?
Turning desalination waste into a useful resource
Tangentially related: there are millions of abandoned oil wells emitting greenhouse gases like methane across the US. Vice had a great video about this last month, but it didn't get much attention here on HN. [1] [2]
Evcxr: A Rust REPL and Jupyter Kernel
I have been using evcxr for the last 2-3 weeks and I'm loving it. There are two things that bother me: (a) fancy UI components seem to only work with Python and C++ kernel (not theoretically, but with available packages), (b) while you can redefine values and functions when you're iterating, you cannot do so with structs (yes, big structs go to a local package that I depend on, but sometimes I have ad-hoc utility structs).
Here's the xeus-cling (Jupyter C++ Kernel) source: https://github.com/jupyter-xeus/xeus-cling/tree/master/src
Do any of the other non-Python Jupyter kernels have examples of working fancy UI components? https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
Jupyter kernels implement the Jupyter kernel message spec. Introspection, Completion: https://jupyter-client.readthedocs.io/en/latest/messaging.ht...
Debugging (w/ DAP: Debug Adapter Protocol) https://jupyter-client.readthedocs.io/en/latest/messaging.ht...
A `display_data` Jupyter kernel message includes a `data` key with a dict value: "The data dict contains key/value pairs, where the keys are MIME types and the values are the raw data of the representation in that format." https://jupyter-client.readthedocs.io/en/latest/messaging.ht...
This looks like it does something with MIME bundles: https://github.com/jupyter-xeus/xeus-cling/blob/00b1fa69d17b...
ipython.display: https://github.com/ipython/ipython/blob/master/IPython/displ...
ipython.core.display: https://github.com/ipython/ipython/blob/master/IPython/core/...
ipython.lib.display: https://github.com/ipython/ipython/blob/master/IPython/lib/d...
You can also run Jupyter kernels in a shell with jupyter/jupyter_console:
pip install jupyter-console jupyter-client
jupyter kernelspec list
jupyter console --kernel python3Ask HN: What is the cost to launch a SaaS business MVP
I interviewed several entrepreneurs, I noticed several spent so much money on developing their product before launch. What is your experience?
When you're not yet paying yourself, your costs are your living costs and opportunity costs (in addition to the given fixed and variable dev and prod deployment cloud costs).
Early feedback from actual customers on an MVP can save lots of development time. GitLab Service Desk is one way to handle emails as issues from users who don't have GitLab accounts.
A beta invite program / mailing list signup page costs very little to set up; you can start building your funnel while you're developing the product.
Cryptocurreny crime is way ahead of regulators and law enforcement
I used to be surprised when I would read articles about this mythical legislative process which can and will protect lowly plebians like me. I'm not sure why so many people are under the impression legislation will be able to keep up with perversely motivated individuals or groups. Who are these legislators with motives so pure that they will never be bribed to only look out for my interests? Let's never forget there is only one person in jail for the 2008 crash this article is trying to convince us was the result of too little regulation. There was never enough incentive to protect us from the people who created that market crash.
I think the problem with 2008 was that no one was doing anything illegal, just a lot of poor judgement and lack of regulations.
Bitcoin was created by people who were so peeved at the regulator's interventions in the '08 crisis that they decided that bypassing the entire financial system was the best option. Forever immortalising "The Times 03/Jan/2009 Chancellor on brink of second bailout for banks" just as a reminder of why they were doing what they did [0].
We don't need better regulation. We need, when financial firms muck up on a global scale, to replace the people who run those firms with different people. There is an easy way to do that, let them go broke.
The biggest problem in the room is that the regulators have bought in to the idea that some companies are "too big to fail". That is a stupid idea, fundamentally unfair, completely throwing out the best part of capitalism which is that recognised idiots aren't allowed to be in charge. That problem will certainly not be solved by giving the regulators more power.
Bitcoin was created in context to "Transparency and Accountability": a campaign motto not coincidentally found in the title of the "Federal Funding Accountability and Transparency Act of 2006".
> The Federal Funding Accountability and Transparency Act of 2006 (S. 2590)[2] is an Act of Congress that requires the full disclosure to the public of all entities or organizations receiving federal funds beginning in fiscal year (FY) 2007. The website USAspending.gov opened in December 2007 as a result of the act
Sen. Obama's office is the origin of this bill; which was fronted by Sens Coburn and McCain, who had the clout.
https://usaspending.gov/ creates a mandatory database with budgetary line item metadata. Where money actually goes is something that is far more transparent and accountable with bitcoin and other public ledgers than any existing ledger covered by bank secrecy laws.
For context, in 2008-09, global financial systems were failing as a result of the American economy: housing bubble burst, HFT "flash crash" that we didn't have CAT or big data tools to determine the cause of, DDOS attacks and cyber security losses increasing YoY, credit default swaps had been rated as AAA securities (they sold bundled bad debt like it was worth something, and then wrote down losses), Enron energy speculation amidst rolling blackouts that were leaving hospitals in the dark, on gas generators, government investments in renewables had been paltry since the Carter administration had put solar panels on the roof of the White House before the whole oil price shock, and oil commodity speculation had driven the price of oil to like 2-4x the 2000 price (with resultant price effects on most CPI inflation/PPP basket goods); but electricity consumption was down in 2008 and renewables hadn't reached production volumes necessary to reach the competitive price point that renewables now present: cheaper than nonrenewables.
Who would have thought that the speculative price would continue to exceed the production cost. incentives or penalties?
Externalities per dollar returned per kWh is one way to assess the total costs of electricity production methods.
"Buy Gold" was the refrain of the day: TV commercials, signs out in front of piano stores (a somewhat-arbitrary commodity, sales of which are observed to be a leading indicator of economic health), signs on the road. And the message was "take your money out the market and put it in gold" which drives up the prices for chips and boards and medical equipment that rely upon that commodity as a material input. Gold is necessary for tropical spec components in high-humidity environments: gold hinges are prized, for example.
But, "look, there's water flowing from the chocolate fountain; so you can go ahead and go" and "you know you want to put it back in there, in that market" we're the appropriate messages given our revenue at the time.
For further technology scene context in 2007-2009,
"Grid computing" links to a number of distributed computing projects: https://en.wikipedia.org/wiki/Grid_computing#History
IIRC, there was a production metric-priced grid system developed around Seattle/Vancouver called "Gold" (?) that was built on Xen and is likely a precursor to metric-priced Cloud services like EC2 and S3 (which now simplify calculations for how much a 51% attack against a Proof of Work txpool with adaptive tx fees costs with n good participants in the game) which incentivize efficiency by penalizing expensive operations.
Code bloat was already a thing: how is everything getting slower when Moore's Law predicts the growth rate in transistor density? Are there sufficient incentives for code efficiency when there seem to be surplus compute resources just idly depreciating.
MySQL primary/secondary replication was considered a viable distributed database system, but securing replication depends upon cert exchange and (optionally), PKI, DNS, and IP tunnels of some sort. And then who has root, write to the journal and tables and indexes on the filesystem, UPDATE, and DELETE access in an inter-organizational distributed systems architecture with XML, Web Services, our very own ESB to scale separately from the database replication and off-site backups that nobody ever checks against the online data, and fragmented and varyingly-implemented industry standards that hopefully specify at least a sufficient key for the record that's unique across ledgers/systems/databases.
BitTorrent DHT magnet: links were extant.
Linden Dollars in Linden Lab's Second Life (there's a price floor on land, which is necessary to sell digital assets/goods/products/services) and accumulated avatar value in e.g. EverQuest and WarCraft (for which there were secondhand markets).
ACH was ACH: GPG-signed files over SFTP on the honor of the audited bank to not allow transfers that deposit money that doesn't exist.
There was no common struct for banking APIs (as apparently only e.g. Plaid, Quicken, and Mint solve for): ledger transactions have a fixed width text field that may contain multiple fields concatenated into one string, and there's no "payee URI" column in the QIF or CSV dumps of an account ledger.
To request more than e.g. the past 90 days of one's own checking account ledger, one was expected to parse tables out of per-month PDFs with e.g. PDFminer at $20 apiece, and then think up ones own natural key in order to merge and lookup records because (2008-01-01,3.99) and (2008-01-01,3.99,storename) are indistinct as a natural key (and when hashed). If you loan a your bank money (for them to now freely invest in the other side, since GLBA in 1999), wouldn't you think that the least they can do is give you `SELECT * WHERE account_id=?` as a free CSV without any datetime limitation in regards to what's offline and what's online.
"Audit the Fed", "Audit DoD" were being chanted by economically-aware citizens amidst severe correction and what was then the most severe recession since the Great Depression: the "Great Recession" it was called, and payouts to essential cronies (who hadn't saved wheat for the famine) were essential.
Overdraft was an error charged to the customer, who didn't build an inconsistent system (CAP theorem) that allows spending money that doesn't exist (at interest charged to the consumer/taxpayer).
"Catch Me If You Can" (2002) described the controls for bank fraud at the time. Why are fees so high?
"Office Space" (1999) described penny-shabing / salami-slicing attack: "fractions of a penny".
"Beverly Hills Ninja" (1997) detailed the story of the Great White Ninja and Tanley! (fistpalm)
"Swordfish" (1999) described a domestic disaster and bank transfers confirmable in seconds.
Ask HN: Why aren't micropayments a thing?
Amazon aws and related services can charge you a rate per email, or per unit time of computation, so why can't news sites just charge you $0.01 to read an article, or even half that?
For reference, there's Web Monetization [1] which tries to solve exactly that.
As others have noted it all boils down to user agent support. Otherwise most publishers probably won't consider giving up ad or subscription financed models.
Also, forcing users into subscriptions allows for better demographics data/statistics.
https://webmonetization.org/ lists Coil (flat $5/mo) as the first Web Monetization provider: https://coil.com/
Web Monetization builds upon ILP (Interledger Protocol), which is designed to work with any type of ledger; though it's probably not possible for any traditional ledger to beat the <1¢ transaction fee that only pre-mined coins have been able to achieve.
I really like their approach and I paid for Flattr for quite a while when it started. But something big needs to push them to the critical adoption rate. On a number of very techy blog posts, I got... 0 from WebMonetization. And that's on best case audience (coming from HN and tech Reddit).
Elon Musk announces $100M carbon capture prize
https://www.xprize.org/prizes/carbon :
> The $20M NRG COSIA Carbon XPRIZE develops breakthrough technologies to convert CO₂ emissions into usable products.
CCS: https://en.wikipedia.org/wiki/Carbon_capture_and_storage
CCU: https://en.wikipedia.org/wiki/Carbon_capture_and_utilization
Sequestration: https://en.wikipedia.org/wiki/Carbon_sequestration
Tim Berners-Lee wants to put people in control of their personal data
Interesting in a technological sense, but what problem it solves isn’t obvious to me. It lets me granularly authorize first party access what data I have in my pod, but there can’t be any technical guarantees with regards to illegitimate sharing or otherwise copying (many might at least cache, for instance) – nor about what is collected and shared outside this system.
I keep seeing data-hubs and identity-providers touting themselves as solutions to the web's privacy issues, but I don't see how they actually solve anything.
It seems like an attempted technical solution to a social problem to me.
The real problem with data based services (ads, Google search, etc) is really that a bunch of data is collected opaquely, unethically, and in some cases illegally. The whole system including data brokers and real time bidding is out of control.
In my (personal) view, it's the technical part of a solution that definitely also needs to have a social/legislative component. It cannot prevent parties from illegitimate sharing of my data, but it does give them the option to hand over control to me. There are lots of companies that currently hold data on us but for whom that data is not their primary competency, and they only need a small nudge (like GDPR) to make having the customer responsible for that data an attractive proposition.
You might be right, but I think it's disingenuous to market it as though this "solves privacy". Worst case, people are lulled into a false sense of security.
Data-storage + authorization doesn't solve any (new) technical privacy-issues; this is "data protection" rather than "data privacy" in my book.
While I recognize the value of W3C LDP and SOLID, I also fail to see anything in SOLID that prevents B from sharing A's now pod-siloed information.
Does it prevent screenshots and OCR?
So it's in standard record structs and that makes it harder for the bad guys?
Who moderates mean memes with my face on them?
It is my hope that future Linked Data spec tutorials model something benign like shapes or cells instead of people: so that we can still see the value.
Laws still exist against things like perjury, even though the existence of the law is not a technical means in itself able to prevent perjury. Note that one of the comments upthread specifically mentioned legislation. The current notion that many people in the tech world have, which roughly states that what determines whether something is kosher is whether it's technically possible to accomplish, is something that needs to change, instead of things just staying a permanent Wild West forever.
There's also an old phrase that putting locks on your doors doesn't actually stop a determined attacker, but that it's okay because they're not meant to—that they're meant to "keep honest people honest". It's a principle that applies here.
No, there are few to no actual privacy improvements over centralized systems.
Perhaps even functional regression: what, are you going to run a hash blocklist across all nodes? Like spamhaus? Is there logging or user accounting? Is anything chain of custody admissable, or are we actually talking about privacy and liberty here?
Is everything just marked, "not for unlimited distribution"? And we dwpend upon there not being bad actors?
Real costs are very different with just friendly early adopters.
Cryptographically signing posts (with LD-Signatures) may help with integrity, but that can be done with centralized systems and does nothing to help with confidentiality.
What about availability? Is it a trivially-DOS'able system?
Governments spurred the rise of solar power
I can't read the article, but I think evidence like this is important when it comes to the whole idea of government subsidises impacting on free markets. The free market has become a bit of a religion for some (you can see in the comments here), and there's this staunch divide between those that believe the free market will always come up with a humanitarian solution, to those that believe the government need to push things for it to happen.
My FIL was head of a large coal plant in the UK, and is very critical of renewable energies, their cost, the impact on the fossil fuel industry etc... It simply isn't possible to have a conversation with him about global warming - he will simultaneously argue it isn't happening and it's natural and not caused by us. He might be an extreme example, but there is general scepticism of government subsidies here.
I'd love to see a study where researchers take things that are seen as good/important/essential to modern life and measure the amount of public/government sponsorship that helped bring it about.
I always find the free market vs government arguments a false dichotomy. Even the free-est markets require a whole host of government services: regulation, standards setting, contracts enforcement, antifraud measures provision and infrastructure to name but a few. People only cry free market when they don't like a particular government activity.
That coal plant relies on infrastructure to get its coal. It needs fair utility prices to deliver its power to customers. It needs reliable (abuse free) markets to sell the power on. It needs voltage and frequency/time standards to access the network. It needs contract enforcement to actually get paid by its customers. It needs insurance and educated workers and a CEO who won't embezzle.
All provided by government, all welcomed by industry. But suddenly government has no place in free markets when it comes to CO2.
Should we prefer penalties or incentives in order to use predictable markets for the change we need?
That depends on whether you want power to be cheaper or more expensive. Cheaper is nice but you'll pay for it taxes etc. Expensive is painful at the point of consumption but encourages efficiency. It's a strategic and political decision and depends on a nations needs and objectives...
> I'd love to see a study where researchers take things that are seen as good/important/essential to modern life and measure the amount of public/government sponsorship that helped bring it about.
Essential technology investment of US tax dollars?
NASA spinoff technologies: https://en.wikipedia.org/wiki/NASA_spinoff_technologies
NSF, DARPA, IARPA, In-Q-Tel, ARPA-e (2009)
List of emerging technologies: https://en.wikipedia.org/wiki/List_of_emerging_technologies
Termux no longer updated on Google Play
The GitHub discussion is significantly more informative and carries a lot of thinking behind the changes: https://github.com/termux/termux-app/issues/1072
IMO a better link than a short paragraph on Wiki.
Note that there are 297 hidden items in that issue so you have to click "Load more..." ceil(297/60) times to read all of the comments about how APK packaging is soon necessary for latest Android devices so the termux package manager can't just dump executable binaries wherever.
FWIU:
- Android Q+ disallows exec() on anything in $HOME, which is where termux installed binaries that may have been writeable by the executing user.
- Binaries installed from APKs can be exec()'d, so termux must keep APK repacks rebuilt and uploaded to a play store.
- Termux shouldn't be installed from Google Play anymore: you should install termux from the F-Droid APK package repos, and it will install APKs instead of what it was doing.
- Compiling to WASM with e.g. emscripten or WASI was one considered alternative. "Emscripten & WASI & POSIX" https://github.com/emscripten-core/emscripten/issues/9479
What about development on-the-device?
- It seems C compiled with clang on the device wouldn't be executable? (If it was, that would be a way around the restriction: distribute packages as source, like the good old days)
> offer users the option of generating an apk wrapping their native code in a usable way. https://github.com/termux/termux-app/issues/1072#issuecommen...
This seems a promising solution: compile from source, create an apk, install - your custom distribution! For popular collections of packages, a pre-built apk.
- Java might be explicitly blocked, being a system language for android, even though its byecode is interpeted and not exec()ed.
- Other interpreted languages should be OK e.g. python
> > offer users the option of generating an apk wrapping their native code in a usable way.
> This seems a promising solution: compile from source, create an apk, install - your custom distribution! For popular collections of packages, a pre-built apk.
FPM could probably generate APKs in addition to the source archive and package types that it already supports.
The conda-forge package CI flow is excellent. There's a bot that sends a Pull Request to update the version number in the conda package feedstock meta.yml when it detects that e.g. there's a new version of the source package on e.g. PyPI. When a PR is merged, conda-forge builds on Win/Mac/Lin and publishes the package to the conda-forge package channel (`conda install -y -c conda-forge jupyterlab pandas`)
The Fedora GitOps package workflow is great too, but bugzilla isn't Markdown by default.
Keeping those APKs updated and rebuilt is work.
Ask HN: What should go in an Excel-to-Python equivalent of a couch-to-5k?
Yesterday, my co-founder published a blog about her experiences Ditching Excel for Python in her job as a Reinsurance Analyst [0].
One of the responses on reddit [1] asked what they should do, "Step 1 day 1," if having read Amy's post they were convinced to try and begin the long journey from tangled Excel/Access spaghetti.
My (flippant) reaction to a friend that brought the comment to my attention was unhelpful; "Step 1 day 1, quit." So he has challenged me to write eight helpful blog posts during the remainder of my Garden Leave.
What should go in them?
[0] https://amypeniston.com/ditching-excel-for-python/
How to write functions in JS / VB script and call them from a cell expression.
How to name variables something other than AB3.
How to use physical units and datatypes. (How to specify XSD datatype URIs that map to native primitives in an additional frozen header row. e.g. py-moneyed and Pint & NumPy ufuncs)
How transitive sort works (is there a tsort to determine what to calculate first (and whether there are cycles) on every modification event?)
Which Jupyter platforms do and don't support interactive charts with e.g. ipywidgets?
pandas.df.plot(kind=) (matplotlib), seaborne (what are the calculated parameters of this chart?), holoviews, plotly, altair
Reproducibility w/ repo2docker / BinderHub:
pip freeze > constraints.txt
cp constraints.txt requirements.txt
conda env export --from-historyAlso,
When is it better to have code in a notebook instead of in a module?
How to export notebook cells to a module with nbdev
How to write tests to assert the quality of the code and the model: @pytest.mark.parametrize, pytest-notebook, jupyter-pytest-2, pytest-jupyter
When is it appropriate to parametrized a notebook with e.g. papermill?
How to handle concurrency: dask.distributed + dask-labextension, ipyparallel
Interesting, your suggestion is that the first thing to do is to learn how to make functions and variable names within excel?
Yeah if you port it to functions and verify that you haven't broken anything, you could then easily port to Python functions that you could call from Excel with an add-on that everyone that opens the sheet needs to have installed; but everyone that opens a sheet that calls Python must have that same extension (and all python package dependencies) installed, too
>My (flippant) reaction to a friend that brought the comment to my attention was unhelpful; "Step 1 day 1, quit." So he has challenged me to write eight helpful blog posts during the remainder of my Garden Leave.
One way to do it would be to do impedance matching: maximizing usefulness by bringing Python to the universe the person who asked the question lives in: Excel, Word, PDF, email.
This means "Automate the Boring Stuff"[0] to ease into programming with something specific to do that is relevant and useful right off the bat for manipulating files, spreadsheets, PDF documents, etc.
Openpyxl[1] for playing more with Excel files, and basically searching "Excel Python" on the internet for more use cases.
Python-docx[2] for playing with Word documents, extracting content from tables, etc.
PdfPlumber[3] for playing with PDFs. Sometimes you have data in .pptx files, and you can use LibreOffice to convert all of them to PDFs because you hate the pptx format:
libreoffice --headless --invisible --convert-to pdf *.pptx
- [0]: https://nostarch.com/automatestuff2
- [1]: https://openpyxl.readthedocs.io/en/stable/
- [2]: https://python-docx.readthedocs.io/en/latest/
Using openpyxl to extract data from exist excel workbooks and other documents would seem like a good place to start.
I suspect many companies use excel workbooks as "forms" with lots of data at the same cell in multiple workbooks.
> I suspect many companies use excel workbooks as "forms" with lots of data at the same cell in multiple workbooks.
Downstream data quality costs can be minimized with data normalized schema and data collection process controls like forms-based data validation.
There are established UI/UX design patterns for data validation of user-supplied data: accessible [web] forms with tab-ordered input fields and specific per-input feedback with accessible HTML5 and ARIA. IIUC, Firefox now supports PDF forms, too?
Why would we move from a spreadsheet to an actual database?
Data integrity:
Referential integrity (making sure that record keys actually point to something when creating, updating, or deleting),
Columnar datatypes (float, decimal, USD, complex fraction),
Access controls (auth(z): authentication and authorization),
Auditing (what was the value before and after that) and Disaster Recovery,
Organizationally-unified schema development and corresponding validation.
Repeatability / Reproducibility: can you replay the steps needed to build the whole sheet? What parameters were entered and how to we script that par so that we can easily assess the relations between the terms of the argument presented?
Scientists turn CO2 into jet fuel
Link to the article in question: https://www.nature.com/articles/s41467-020-20214-z
They call it 'renewable' yet use Iron as a catalyst. I'm not a chemist - how is that renewable?
Yao, B., Xiao, T., Makgae, O.A. et al. "Transforming carbon dioxide into jet fuel using an organic combustion-synthesized Fe-Mn-K catalyst." Nat Commun 11, 6395 (2020). https://doi.org/10.1038/s41467-020-20214-z
Synthetic biology can do it better. Fix CO2 by growing sugar cane, turn sugar into jet fuel with genetically engineered yeast.
https://www.total.com/media/news/press-releases/total-and-am...
Still not commercially viable, but much closer to it than the linked process.
That approach requires lots of land, water, and time. Nothing wrong with that, but much of the funding for CO2->jet fuel research comes from the military, which wants to be able to create liquid fuel quickly from CO2 in the atmosphere and they don't care how inefficient it is. Military customers are typically not very interested in processes that require running a farm.
The military also spends a lot of time turning jet fuel into electricity, because the former is energy-dense, easy to store, easy to transport. And they need electricity in lots of places without a grid.
Where would the reverse be useful? Somehow you have unlimited electricity but no fuel, and you have time and space to run a chemical refinery? (And isn't carbon capture from the atmosphere likely to require farm-scale infrastructure anyway?)
You cannot run aircraft on electricity; liquid fuel is the only option. The military is keenly interested in being able to refuel aircraft in situations where liquid fuel shipments might be denied or unavailable for some reason.
You can run aircraft on electricity.
Locomotives run on electrical energy produced by diesel generators (because electric motors are more energy efficient), for example.
The limits are the cost and weight of the batteries and the charge time.
You can't in any practical sense. Locomotives don't have a problem carrying the weight of a diesel engine plus a generator; in fact the extra weight works in their favor.
If you tried that trick with an airplane it would never get off the ground. Locomotives use this system not for efficiency but because they need the low-end torque electric motors provide. (If they only needed efficiency they'd just drive the wheels with the diesel engine directly.) Airplanes don't need torque; they need power.
You can run an airplane on batteries but only for a few minutes with modern battery technology. Practical battery-powered aircraft for military applications are not going to happen any time soon. We might see battery-powered air taxis soon but those are not even close to meeting military requirements for fighters, transports, etc.
From https://en.wikipedia.org/wiki/Electric_aircraft :
> [For] large passenger aircraft, an improvement of the energy density by a factor 20 compared to li-ion batteries would be required
The time it takes to surpass this energy density threshold is affected by battery tech investments; which had been comparatively paltry in terms of defense spending. Trillions on batteries would've been a much better investment; with ROI.
Sadly, some folks in defense still can't understand why non-oil investments in battery tech are best for all.
There are multiple electric trainer aircraft with flight times over an hour and quite a few more in development.
Jet engines are terribly inefficient (30-50% efficient) compared to electric motors.
Show HN: Stork: A customizable, WASM-powered full-text search plugin for the web
OP - congratulations on shipping! WASM-powered search caught my eye.
Looks like the search index is downloaded and used locally in the browser, so this is as fast as search can get. One trade-off though is that you're limited to relatively small datasets. While this shouldn't be an issue for small-medium static sites, an index that needs to be downloaded over the wire will affect your page performance for larger sites / datasets.
In those cases, you'd want to use a dedicated search server like Algolia or its open source alternative Typesense [1]. Both of them store the index server-side and only return the relevant records that are searched for.
For eg: you'd probably not want to download 2M records over the wire to search through them locally [2]. You'd be better off storing the index server-side.
[1] https://github.com/typesense/typesense (shameless plug)
Just a quick non-thought-through idea but would it be possible to build an index in a way that allows clients to download only parts of it based on what they search? I.e. the search client normalizes the query in some way and then requests only the relevant index pages. The index would probably be a lot larger on the server but if disk space is not an issue...?
(Though at some point you have to ask yourself what the benefits of such an approach are compared to a search server.)
Merkle Search Trees: Efficient State-Based CRDTs in Open Networks https://hal.inria.fr/hal-02303490/document
Peer-to-Peer Ordered Search Indexes https://0fps.net/2020/12/19/peer-to-peer-ordered-search-inde... (which adds useful context about the above)
Sorry for just dropping these links, I should already be asleep :)
> Merkle Search Trees: Efficient State-Based CRDTs in Open Networks https://hal.inria.fr/hal-02303490/document
https://scholar.google.com/scholar?cites=7160577141569533185... ... "Merkle Hash Grids Instead of Merkle Trees" (2020) https://scholar.google.com/scholar?cluster=13503894708682701...
Browser-side "Blockchain Certificate Transparency" applications need to support at least exact key lookup by domain/SAN and then also by cert fingerprint value; but the whole CT chain with every cert issue and revocation event is impractically large in terms of disk space.
https://github.com/amark/gun#history may also be practically useful.
Upptime – GitHub-powered open-source uptime monitor and status page
Maker here, I made Upptime as a way to scratch my own itch... I wanted a nice status page for Koj (https://status.koj.co) and was previously using Uptime Robot which didn't allow much customization to the website.
It started with the idea of using GitHub issues for incident reports and using the GitHub API to populate the status page. The obvious next step was opening the issues automatically, so the uptime monitor was born. I love using GitHub Actions in interesting ways (https://github.blog/2020-08-13-github-actions-karuna/) and have made it a large part of my toolkit!
https://news.ycombinator.com/item?id=25557032 mentions "~3000 minutes per month". GitLab's new pricing structure: [(runner_minutes, usd_per_month), (400, $0), (2_000, $4), (10_000, $19), (50_000, $99)]
You can run a self-hosted GitHub or GitLab Runner with your own resources: https://docs.github.com/en/free-pro-team@latest/actions/host...
GitLab [Runner] also runs tasks on cron schedules.
The process invocation overhead for CI is greater than for a typical metrics collection process like a nagios check or a memory-resident daemon like collectd with the curl plugin and the "Write HTTP" plugin (if you're not into using a space and time efficient timeseries database for metrics storage)
An open source project with a $5/mo VPS could run collectd in a container with a config file far far more energy efficiently than this approach.
Collectd curl statistics: https://collectd.org/documentation/manpages/collectd.conf.5....
Collect list of plugins: https://collectd.org/wiki/index.php/Table_of_Plugins
Is there a good way to do {DNS, curl HTTP, curl JSON} stats with Prometheus (instead of e.g. collectd as a minimal approach)?
Show HN: Simple-graph – a graph database in SQLite
I wonder if there are ways, in SQLite, to build indices for s,p,o/s,p/p,o/ and maybe more subtle ones... That would be uber nice, given the fact that most graph databases have their own indexing strategies, and you cannot craft your own.
rdflib-sqlalchemy is a SQLAlchemy rdflib graph store backend: https://github.com/RDFLib/rdflib-sqlalchemy
It also persists namespace mappings so that e.g. schema:Thing expands to http://schema.org/Thing
The table schema and indices are defined in rdflib_sqlalchemy/tables.py: https://github.com/RDFLib/rdflib-sqlalchemy/blob/develop/rdf...
You can execute SPARQL queries against SQL, but most native triplestores will have a better query plan and/or better performance.
Apache Rya, for example:
> indexes SPO, POS, and OSP.
Thanks for your comment. I use rdflib frequently but have never tried the SQLAlchemy back end. Now I will. That said, Jena or Fuseki, or the commercial RDF stores like GraphDB, Stardog, and Allegrograph are so much more efficient.
In CPython, types implemented in C are part of the type tree
The docs should have coverage on this:
Python/C API Reference Manual: https://docs.python.org/3/c-api/index.html
Python/C API Reference Manual » Object Implementation Support > Type Objects: https://docs.python.org/3/c-api/typeobj.html
CPython Devguide > Exploring Python Internals > Additional References: https://devguide.python.org/exploring/
Experiments on a $50 DIY air purifier that takes 30s to assemble
From "Better Box Fan Air Purifier" https://tombuildsstuff.blogspot.com/2013/06/better-box-fan-a... :
> Air purifiers can be expensive and you've probably seen articles recommending to just put a 20" x 20" x 1" furnace filter on a cheap 20" box fan and POOF! instant cleaner air for not a lot of money. It really does clean the air pretty cheap.
> There's a problem with this though. These fans weren't designed to be run with a filter. The filter will restrict air flow which will put a higher strain on the motor causing it to use more electricity and in worse cases could be a fire hazard. The higher the MERV rating (cleaning efficiency) of the filter the more stress it will put on the fan.
> Don't worry! You can still have your cheap air purifier as long as the filter area is increased to decrease the effect of air resistance. Instead of using one 20x20x1 filter we'll use two 20x25x1 filters which increases the filter surface area over 250%. It's a little more expensive because you're using two filters instead of one but the increased filter surface area also helps the filter last longer before it gets clogged up and we're saving on energy use compared to a single filter.
> There's a problem with this though. These fans weren't designed to be run with a filter. The filter will restrict air flow which will put a higher strain on the motor causing it to use more electricity and in worse cases could be a fire hazard. The higher the MERV rating (cleaning efficiency) of the filter the more stress it will put on the fan.
I'm fairly certain the opposite is true. In fact, most rowing machines work on this principle. The easiest setting is the one where the fan is as closed off as possible because it's pulling a vacuum. Less air, less resistance, easier to row / less power required to spin the fan. If anything, fans should draw less current with air flow on the inlet side restricted.
A. Putting two [larger] filters in a 'V' with cardboard to fill the top and bottom pulls the same amount of air through a larger area of filters
B. pulling the same volume of air through greater surface area results in greater pressure between the filter and fan than one filter directly affixed to the fan
C. The lower air pressure / "suction" due to an obstructed intake causes an electric fan motor to fail more quickly.
D. Increasing the air pressure that the motor is in reduces the failure rate?
Goodreads plans to retire API access, disables existing API keys
This makes absolutely no sense and has no relation to any economic variables. Goodreads isn’t some struggling self-funded startup — it’s owned by Amazon.com. The acquisition was a deal that should have never been approved, if the Obama administration had been anything beyond completely impotent at protecting us from monopoly games:
https://www.theguardian.com/books/2013/apr/02/amazon-purchas...
I would like to understand the true strategic interest behind this. Is Amazon simply penny-pinching now that they’ve successfully obliterated the market for both new and used books online? There’s way more to this story than appears on the surface.
It's not mentioned in the article but Amazon had disabled their affiliate link program for Goodreads ahead of this announcement, which cut off a major source of their revenue for them, and forced them to sell. They had no choice.
The strategic reason for Amazon is obvious. As someone else mentioned, Amazon doesn't want Goodreads data to be used to add value to their competitors' offerings.
Speaking as a developer who tried to build on top of Goodreads API, I also want to add that this was a long time coming. The API had been neglected for some time. And some of the most interesting datasets weren't even made available through the API.
Apparently Amazon’s Kindle lost the ability to share progress on Goodreads in the last week as well.
I guess the writings on the wall...
Turing Tumble Simulator
(Turing Tumble is a (fun) marble-powered computer game.)
Python Pip 20.3 Released with new resolver
If anyone hasn't seen it, now is a good time to look at https://python-poetry.org/ It is rapidly becoming _the_ package manager to use. I've used it in a bunch of personal and professional projects with zero issues. It's been rock solid so far, and I'm definitely a massive fan.
I occasionally ask our principle / sr. Python engineers about this, and their response is always, "These things come and go, virtualenv/wrappers + pip + requirements.txt works fine - no need to look at anything else."
We've got about 15 repos, with the largest repo containing about 1575 files and 34MBytes of .py source, 14 current developers (with about 40 over the last 10 years) - and they really are quite proficient, but haven't demonstrated any interest at looking at anything outside pip/virtualenv.
Is there a reason to look at poetry if you've got the pip/virtualenv combination working fine?
People who use poetry seem to love it - so I'm interested in whether it provides any new abilities / flexibility that pip doesn't.
beware of zealot geeks bearing gifts. if your environment is currently working fine and you are only interested in running one version of python and perhaps experimenting with a later one then venv + pip is all you need, with some wrapper scripts as you say to make it ergonomic (to set the PYTHON* environment variables for your project, for example)
The main reason I use Poetry is for its lockfile support. I've written about why lockfiles are good here: https://myers.io/2019/01/13/what-is-the-purpose-of-a-lock-fi...
Pip supports constraints files. https://pip.pypa.io/en/stable/user_guide/#constraints-files :
> Constraints files are requirements files that only control which version of a requirement is installed, not whether it is installed or not. Their syntax and contents is nearly identical to Requirements Files. There is one key difference: Including a package in a constraints file does not trigger installation of the package.
> Use a constraints file like so:
python -m pip install -c constraints.txtWhat does that command do? Does it install dependencies, or simply verify the versions of the installed dependencies?
It just gives you an error:
ERROR: You must give at least one requirement to install (see "pip help install")
Your package manager should be boring, extremely backward and forward compatible, and never broken. Experience has shown this not to be true for python. Several times over the years i’ve found myself, pinning, upgrading, downgrading, or otherwise juggling versions of setuptools and pip in order to work around some bug. Historically I have had far more problems with the machinery to install python packages I have had with all of the other python packages being installed combined, and that is absurd.
Convolution Is Fancy Multiplication
FWIW, (bounded) Conway's Game of Life can be efficiently implemented as a convolution of the board state: https://gist.github.com/mikelane/89c580b7764f04cf73b32bf4e94...
How to better ventilate your home
I recently purchased a home and spent some time improving ventilation in it this year. This is a home in Minneapolis, so just "opening a window" is not an option, as it would be too cold in the winter and dramatically reduce energy efficiency. Its effectiveness also changes based on factors like the wind, and the temperature difference between inside and out.
The general strategy for doing this right now with existing homes is to insulate your home as well as possible for energy efficiency, then install a continuous ventilation fan. This is essentially a bathroom fan, except that it runs all the time at a constant low speed, helping to circulate the air through and out your house by "pulling" a designed amount of air through the cracks.
I didn't want to punch a new hole in the house just to do this, and already had a bathroom fan installed, so as a hack I just turned it into a whole house ventilation fan with this: https://www.aircycler.com/pages/smartexhaust
Basically you calculate how much CFM you need per hour based on the square footage of your house, and then you set it on the fan control. It still acts like a bathroom fan, except every hour it also runs for a set period of time (in my case, about 12 minutes).
The standard for this is ASHRAE 62.2. Use this to calculate the CFM for code: https://homes.lbl.gov/ventilate-right/step-3-whole-building-...
Then the formula for calculating the fan run time is on this sheet: https://cdn.shopify.com/s/files/1/0221/7316/files/AC_DOC_7_0...
And presto, you have a ventilation system without having to do a lot of work. Do _not_ try this with a crappy, rusty old bathroom fan - clean or replace the motor first, and for extra credit, use an arc fault circuit interrupter on the breaker so if the motor fails it will blow the fuse instead of potentially causing a fire.
Note: This is really just to manage general air quality and VOCs. If you want to specifically make ventilation for COVID-19, that's a different problem. They focus on Air Changes per Hour (ACH), and a cubic feet calculation is used rather than square feet. There's no "recommended amount" of ACH for managing COVID-19. You're likely improving the situation by increasing it, but I wouldn't start inviting people over after you did it. ACH is very high in ICUs but staff are still getting sick there.
RE Humidity - ideal range varies based on region and outside temperature, but this chart roughly shows it: https://lh3.googleusercontent.com/proxy/mPz-jGdLpnPGgvWR3Egf...
I have an on-furnace humidifier controlled by an ecobee for the winter, it's a huge quality of life improvement if you live in cold climates, but make sure to set it to "frost control" otherwise it won't lower the humidity based on the outside air and you can get mold in your walls. For summer, a standard house A/C combined with continuous ventilation should be sufficient to bring down humidity levels.
Finally, the "correct" air ventilation is a moving target with trade-offs and concerns of the moment. It was higher in 1925 (30cfm/person) to try to prevent tuberculosis and infectious diseases, then was lowered to 5cfm/person in the 70s during the energy crisis, and is currently at 15cfm/person. I imagine COVID-19 could make us re-consider the current recommendations. https://homes.lbl.gov/ventilate-right/ashrae-standard-622
I'm in an older home that leaks a sieve but has central air. Would turning on that house fan to circulate without running the AC accomplish the same thing? Also, do any smart thermostats allow this sort of thing to be automated?
"Fan control with a Nest thermostat" https://support.google.com/googlenest/answer/9296419?hl=en
Looks like there could be: (1) an every hour for n minutes schedule; (2) an option to run the fan with the thermostat off; (3) an option to shut off the fan when everyone is gone
Quantum-computing pioneer Peter Shor warns of complacency over Internet security
If an organization has a 5 year refresh cycle (~time to implement a new IT system), and there exists a quantum computer with a sufficient number of error-corrected qubits by 2027 [1], an organization/industry has 5 years from 2022 to go quantum-resistant: replace their existing solution with quantum-resistant algos (and, in some cases, a DLT with a coherent pan-industry API) and/or double their RSA and ECDSA key sizes.
[1] "Quantum attacks on Bitcoin, and how to protect against them (ECDSA, SHA256)" https://news.ycombinator.com/item?id=15907523
Which DLT/blockchains without PKI (or DNS) will implement the algorithms selected from the NIST Post-Quantum Cryptography (PQC) round 3 candidate algorithms? https://csrc.nist.gov/projects/post-quantum-cryptography
CERN Online introductory lectures on quantum computing from 6 November
This is probably a dumb question but are there any data sciencey or tensorflowy things that can be done faster on a quantum computer?
There are some basic linear algebra subroutines (Matrix inversion, finding eigenvalues & eigenvectors) that can be performed with an exponential speedup on a quantum computer in theory, that's why there is so much interest in Quantum Machine Learning. If you are asking about the current hardware level, then no, current quantum computers can not solve any practical problem faster than a classical computer.
In theory classical computers are also not limited to have better solutions to problems where quantum computers claim superiority like prime factorisation or like Tang's quantum inspired classical algorithm that beats HHL for low rank matrices.
> There are some basic linear algebra subroutines (Matrix inversion, finding eigenvalues & eigenvectors) that can be performed with an exponential speedup on a quantum computer in theory
Eh... those particular subroutines have polynomial time algorithms already on a classical computer.
You can't exponentially speed up something that's polynomial time already.
https://quantumalgorithmzoo.org/ lists algorithms, speedups, and descriptions.
That's a great list, thanks!
(Though for the benefit of readers here, the list doesn't include any "basic linear algebra subroutines (Matrix inversion, finding eigenvalues & eigenvectors)").
A Manim Code Template
The demo video looks cool. It's maybe not obvious that there's a link to the code-video-generator (which is built on manim by 3blue1brown) demo video in the README: https://youtu.be/Jn7ZJ-OAM1g
Startup Financial Modeling: What is a Financial Model? (2016)
https://www.causal.app/ has free business model templates: SaaS (Foresight), eCommerce (https://foresight.is/), Startup Runway, Buy/Rent, Ads Calculator
The article explicitly, repeatedly says templates are a bad idea. If you disagree, it'd be interesting to see your reasoning, rather than links to free templates.
We'd do better to find a list of business modeling books and tools.
And then take a look at integrating actual data sources; hopefully some quantitative with APIs.
Uncertainties supports mean±"error" w/ "error propagation": https://pypi.org/project/uncertainties/
Sliders etc can be done in Jupyter notebooks with e.g. ipywidgets: https://ipywidgets.readthedocs.io/en/latest/
At what grade level do presidential candidates debate?
Intelligence does not imply superior moral, ethical, or rational judgement.
Incomplexity of speech does not imply lack of intelligence.
Here's the section on Simple English in Simple English Wikipedia: https://simple.wikipedia.org/wiki/Wikipedia:About#Simple_Eng...
Imagine being reprimanded for use of complex words and statistical terms in an evidence-based policy discussion in a boardroom. Imagine someone applying to be CEO, President, or Chairman of the Board and showing up without a laptop, any charts, or any data.
Topicality!
Perhaps there is a better game for assessing competency to practice evidence-based policy.
This commenter effectively refutes the claim that Fleisch-Kincaid is a useful metric for assessing the grade-level of interpretively-punctuated spoken language: https://news.ycombinator.com/item?id=24807610
Like I said, from "Ask HN: Recommendations for online essay grading systems?" https://news.ycombinator.com/item?id=22921064 :
> Who else remembers using the Flesch-Kincaid Grade Level metric in Word to evaluate school essays? https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readabi...
> Imagine my surprise when I learned that this metric is not one that was created for authors to maximize: reading ease for the widest audience is not an objective in some deparments, but a requirement.
> What metrics do and should online essay grading systems present? As continuous feedback to authors, or as final judgement?
That being said, disrespectful little b will not be tolerated or venerated by the other half of the curve.
ElectricityMap – Live CO₂ emissions of electricity production and consumption
What's going on in Queensland, Australia? "793g Carbon intensity" seems insanely high, especially as compared with Norway/France/Sweden (at 24-41g carbon intensity).
Does anyone have insight into this? I've never pictured Australia as a big contributor to pollution.
Good observation. I was curious, and it looks like the parser code to retrieve the data is open source; so here's the relevant module for Australia:
https://github.com/tmrowco/electricitymap-contrib/blob/24ea2...
The production mix (energy generation by source) source URL is listed on L326.
Opening the dataset in LibreOffice Calc, filtering by Region = QLD1, filtering for Current Output > 0, and sorting by Current Output descending does seem to show that the vast bulk of production reported has fuel source descriptor 'Black Coal'.
If that's a bug in the reported data or source, hopefully the ElectricityMap team can jump in to take a look. The data initially seems legit, from searching for some of the power stations listed.
Edit / addendum: worth searching for Australia in the project's GitHub issues too; there's some existing (although not directly related) investigation & validation work ongoing for that module
Also - home solar penetration in QLD is the highest in Australia. Is that not factored in to the equation? It doesn't look that way given 0% renewables shown.
This website is pulling information from public sources, so it can only show public generation & consumption information.
If someone is running a grid-tied solar system, outbound energy might be accounted for in these statistics, but it won't show any immediate consumption of that solar generation by the property, nor will it show any battery storage, it's only capable of showing what goes through the energy meter.
+1. California has substantial rooftop solar (11 Gwh peak) behind the meter, which isn’t shown in these graphs.
https://pv-magazine-usa.com/2019/04/15/californias-solar-pow...
How would behind the meter electricity consumption change the reported amount of CO2 emitted by other electricity production sources?
The map is intended to show the emissions intensity of electricity so if you miss a bunch of electricity from distributed generation you will over-estimate the intensity.
Man that was a dumb question. Thanks for clarifying.
So we'd need electric utility companies to share the live data of how many kwh of solar and wind people are selling back to the grid in order to get an accurate regional comparison of real-time carbon intensity?
FWIU, they're already parsing the EIA data; but it's significantly more delayed than the max 2 hour delay specified by ElectricityMap.
Here's the parser for the current data from EIA: https://github.com/tmrowco/electricitymap-contrib/blob/maste...
Should the EIA (1) source, aggregate, cache, and make more real-time data available; and (2) create a new data item for behind the meter kwh from e.g. residential wind and solar?
(Edit) "Does EIA publish data on peak or hourly electricity generation, demand, and prices?" https://www.eia.gov/tools/faqs/faq.php?id=100&t=3
> Hourly Electric Grid Monitor is a redesigned and enhanced version of the U.S. Electric System Operating Data tool. It incorporates two new data elements: hourly electricity generation by types of energy/fuel source and hourly sub-regional demand for certain balancing authorities in the Lower 48 states.
> [...]
> EIA does not publish hourly electricity price data, but it does publish wholesale electricity market information including daily volumes, high and low prices, and weighted-average prices on a biweekly basis.
AFAIU, retail intraday rates aren't yet really a thing in the US; but some countries in Europe do have intraday rates (which create incentives for the grid scale energy storage necessary for wide-scale rollout of renewables).
(Edit) "Introduction to the World of Electricity Trading" https://www.investopedia.com/articles/investing/042115/under... :
> Energy prices are influenced by a variety of factors that affect the supply and demand equilibrium. On the demand side, commonly referred to as a load, the main factors are economic activity, weather, and general efficiency of consumption. On the supply side, commonly referred to as generation, fuel prices and availability, construction costs and the fixed costs are the main drivers of the price of energy. There's a number of physical factors between supply and demand that affect the actual clearing price of electricity. Most of these factors are related to the transmission grid, the network of high voltage power lines and substations that ensure the safe and reliable transport of electricity from its generation to its consumption.
Which customers (e.g. data centers, mining firms) would take advantage of retail intraday rates?
How does cost and availability of storage affect the equilibrium price of electricity?
>So we'd need electric utility companies to share the live data of how many kwh of solar and wind people are selling back to the grid in order to get an accurate regional comparison of real-time carbon intensity?
You actually need production numbers for what's used "behind the meter" which by its nature is not directly available live or otherwise and has to be estimated.
>Which customers (e.g. data centers, mining firms) would take advantage of retail intraday rates?
Loads! In the UK you can get half-hourly priced electricity even at a domestic level (with a smart meter) and if you have loads that can be be re-scheduled (mainly EV charging but if you had an electric storage heater that would work as ell) you can save quite a lot of money.
Heavy industrial users definitely move usage around both to avoid expensive wholesale charges but also to reduce their transmission connection charges which (in the UK) are based on their usage during the most congested periods of the year. Water companies will vary pump operations for this reason and water pumping and treatment alone is 2% of electricity use.
Data centers definitely pay on a half-hourly settled basis but tend not to shift their workloads around to take advantage, some data centers will run their cooling systems in such a way as to reduce usage during the most expensive few half hours though. I have heard that larger users like Amazon, FB, Google, will automatically load balance between global centers to reduce electricity bills and carbon footprint.
> how many kwh of solar and wind people are selling back
Not just selling back. Producing - and then either using or selling.
This USA data seems kind of strange to me. For instance my home state, Kansas, we are fourth in the nation for installed wind capacity, supplying 41% of our energy demand. Our sole nuclear power plant provides an 18% base load. 33% is coal and the rest random crap. We're very population sparse and low population in general. Yet in other areas of the country that are strictly coal and natural gas fired, with a greater population density and higher total population, they're less turd colored. I'm guessing this is a strange interaction between renewable availability or it just might be a lack of available data.
From https://github.com/tmrowco/electricitymap-contrib#data-sourc... :
> Here are some of the ways you can contribute:
> Building a new parser, Fixing a broken parser, Changes to the frontend, Find data sources, Verify data sources, Translating electricitymap.org, Updating region capacities
I sent a few tweets and emails about the data in this region but nothing happened here either
Bash Error Handling
From https://twitter.com/b0rk/status/1312413117436104705 :
> TIL that you can use the "DEBUG" trap to step through a bash script line by line
trap '(read -p "[$BASH_SOURCE:$LINENO] $BASH_COMMAND?")' DEBUG
>> you can also customize the prompt with set -x
export PS4='+(${BASH_SOURCE}:${LINENO}) '
set -x
A Customer Acquisition Playbook for Consumer Startups
> For consumer companies, there are only three growth “lanes” that comprise the majority of new customer acquisition:
> 1. Performance marketing (e.g. Facebook and Google ads)
> 2. Virality (e.g. word-of-mouth, referrals, invites)
> 3. Content (e.g. SEO, YouTube)
> There are two additional lanes (sales and partnerships) which we won't cover in this post because they are rarely effective in consumer businesses. And there are other tactics to boost customer acquisition (e.g PR, brand marketing), but the lanes outlined above are the only reliable paths for long-term and sustainable business growth.
Marketing calls those "channels". I don't think they're exclusive categories: a startup's YouTube videos could be supporting a viral marketing campaign, for example; ads aren't the only strategy for (targeted) social media marketing; if the "ask" / desired behavior upon receiving the message is to share the brand, is that "viral"?
What about Super Bowel commercials?
Traditional marketing: press releases, (linked-citation-free) news wires, quasi-paid interviews, "news program" appearances, product placement.
"Growth hacking": https://en.wikipedia.org/wiki/Growth_hacking
Gathering all open and sustainable technology projects
Great list! https://github.com/protontypes/awesome-sustainable-technolog...
"Earth" topic label: https://github.com/topics/earth
Though there are unfortunately zero (0,) results for "sustainability-report[s,ing]", it may be useful to have a heading: https://github.com/topics/sustainability-reporting ... GRI Reporting Standards & XBRL: https://en.wikipedia.org/wiki/Global_Reporting_Initiative , Sustainability Cloud, https://en.wikipedia.org/wiki/Sustainability_reporting
TIL about the "SWEET" (Semantic Web for Earth and Environmental Terminology) Ontologies: https://github.com/ESIPFed/sweet
SunPy: https://docs.sunpy.org/en/stable/generated/gallery/index.htm...
Jupyter Notebooks Gallery
This is not a valid Show HN, so we've taken that out of the title. Please see the rules: https://news.ycombinator.com/showhn.html. Note the word 'lists'.
Lists tend not to make good HN submissions because the only thing one can really discuss about them is the lowest common denominator (or greatest common factor?) of the list elements. It would be better to pick one of the most interesting elements and submit that instead.
HN is itself a list, and a pointer to a pointer to a pointer is too much indirection.
https://hn.algolia.com/?query=denominator%20list%20by:dang&d...
Jupyter/Jupyter > Wiki > "A gallery of interesting Jupyter Notebooks" lists hundreds of notebooks: https://github.com/jupyter/jupyter/wiki/A-gallery-of-interes...
The mybinder.org Grafana dashboard lists the most popular notebook repos in the last hour: https://grafana.mybinder.org/
Jupyter/Nbviewer > FAQ > "How do you choose the notebooks featured on the nbviewer.jupyter.org homepage?" :
> We originally selected notebooks that we found and liked. We are currently soliciting links to refresh the home page using a Google Form. You may also open an issue with your suggestion.
https://nbviewer.jupyter.org/faq#how-do-you-choose-the-noteb...
Google Form: https://docs.google.com/forms/d/e/1FAIpQLSd6AlVvC7KagENypGTc...
Here's the Nbviewer source code. AMP (And https://schema.org/ScholarlyArticle / Book / CreativeWork metadata) could be useful. https://github.com/jupyter/nbviewer
Jupyter notebooks are almost a huge step forward for programming. The instant feedback, saving of partial program state, and visualizations mixed in are amazing and fun to work with. The fact that it's taking place in a browser, you don't have modal editing with Vim, you don't get plugin support (like MyPy), and you don't have intellisense is a huge step backwards.
I'm hoping that someone addresses these issues, I know VSCode is trying, and Scala "Metals" worksheets are another similar concept (though different, it puts variable values in comments next to code as you update the code).
I feel like all of these are a step closer than what the Lighttable/Eve teams were shooting for, but not quite the next generation way of coding that it could be.
I'm a huge fan of rmarkdown since it addresses many of these shortcomings by way of being much simpler than jupyter notebooks. The files are simply text files, and you can write python too if you like, they aren't just for R.
NestedText, a nice alternative to JSON, YAML, TOML
So, this is basically YAML, "but better". I can repeat once more that "easily understood and used by both programmers and non-programmers" is unapologetically stupid concept that can never succeed. So I see how all of this will sound all too familiar to anybody with a little experience, which makes them to automatically dismiss this YAYAML.
But YAML is really quite complicated, and JSON (which shouldn't be used for config files at all) and TOML (which I love and wish it would gain more popularity) aren't exactly alternatives to YAML. So, I would be actually totally ok with "YAML, but better", as a way to deprecate YAML.
Now, it is clear from the start that this cannot deprecate YAML, because it doesn't even have booleans and numbers. But, surprisingly, I can accept this as well: ok, let's just assume that being good at dealing with strings may be enough.
The problem is, it isn't clear at all from the docs, if this is better than YAML at anything. It raises dozens of questions. I'll start with the most basic ones (using [] as a wrapper/delimiter): how do I represent values [ a], [a ], ["a"] and [""] in this file format?
What kind of values are [<whitespace>a] and [a<whitespace>] supposed to be? They look like typical YAML syntax traps to me.
Why should JSON never be used for configuration? It is sufficient for declaratively expressing anything I have encountered. Do we really need references or other stuff from YAML? For configuration this seems unnessecary, provided that the program, which interprets the result of parsing the JSON is well written.
JSON lacks comments and will fail for a missing or extra comma, so it's not great for configuration written by humans.
You can use HJSON which is the json with comments. It's fully compatible with json so easy to introduce into anything that does json. https://hjson.github.io/
JSON5 also supports comments and multiline strings with `\`-escaped newlines: https://json5.org/
Triple-quoted multiline strings like HJSON would be great, too.
From "The description of YAML in the README is inaccurate" https://github.com/KenKundert/nestedtext/issues/10 :
> I will mention something else. The section about the "Norway problem" is not quite accurate. Some YAML loaders do in fact load no as false. These are usually YAML 1.1 loaders. YAML 1.2's default schema is the same as JSON's (only true, false, 'null and numbers are non-strings).
> Any YAML loader is free to use any schema it wants. That is, no loader is required to to load no as false. Good loaders should support multiple schemas and custom schemas. The Norway problem isn't technically a YAML problem but a schema problem.
> imho, YAML's biggest failing to date is not making things like this clear enough to the community.
> Note: PyYAML has a BaseLoader schema that loads all scalar values as strings.
Algorithm discovers how six molecules could evolve into life’s building blocks
It'd be nice if they could do the same for protein synthesis, but that's obviously a much harder problem.
Folding@home https://en.wikipedia.org/wiki/Folding@home :
> Folding@home (FAH or F@h) is a distributed computing project aimed to help scientists develop new therapeutics to a variety of diseases by the means of simulating protein dynamics. This includes the process of protein folding and the movements of proteins, and is reliant on the simulations run on the volunteers' personal computers.
"AlphaFold: Using AI for scientific discovery" (2020) https://deepmind.com/blog/article/AlphaFold-Using-AI-for-sci...
https://www.kdnuggets.com/2019/07/deepmind-protein-folding-u... :
> At last year’s Critical Assessment of protein Structure Prediction competition (CASP13), researchers from DeepMind made headlines by taking the top position in the free modeling category by a considerable margin, essentially doubling the rate of progress in CASP predictions of recent competitions. This is impressive, and a surprising result in the same vein as if a molecular biology lab with no previous involvement in deep learning were to solidly trounce experienced practitioners at modern machine learning benchmarks.
Citations of "Resource-efficient quantum algorithm for protein folding" (2019) https://scholar.google.com/scholar?cites=1037213034434902738...
Protein folding: https://en.wikipedia.org/wiki/Protein_folding
I always wonder how many open research problems could benefit from a working hand from computing. Sometimes I wish research would be as available as an open-source project.
As a computer scientist who worked in a physics lab: a lot. Other sciences could desperately use the help of talented computer scientists. Unfortunately those other sciences are mostly mired in academia nonsense and don't pay well.
"Applied CS"
Computational science: https://en.wikipedia.org/wiki/Computational_science
Computational biology: https://en.wikipedia.org/wiki/Computational_biology
Computational thinking: https://en.wikipedia.org/wiki/Computational_thinking :
> The characteristics that define computational thinking are decomposition, pattern recognition / data representation, generalization/abstraction, and algorithms.
Additional skills useful for STEM fields: system administration / DevOps / DevSecOps, HPC: High Performance Computing (distributed systems, distributed algorithms, performance optimization; rewriting code that is designed to test unknown things with tests and for performance), research a graph of linked resources and reproducibly publish in LaTeX and/or computational notebooks such as Jupyter notebooks, dask-labextension, open source tool development (& sustainable funding) that lasts beyond one grant
Physicists build circuit that generates clean, limitless power from graphene
"Fluctuation-induced current from freestanding graphene" (2020) https://journals.aps.org/pre/abstract/10.1103/PhysRevE.102.0... https://doi.org/10.1103/PhysRevE.102.042101
> In the 1950s, physicist Léon Brillouin published a landmark paper refuting the idea that adding a single diode, a one-way electrical gate, to a circuit is the solution to harvesting energy from Brownian motion. Knowing this, Thibado's group built their circuit with two diodes for converting AC into a direct current (DC). With the diodes in opposition allowing the current to flow both ways, they provide separate paths through the circuit, producing a pulsing DC current that performs work on a load resistor.
> Additionally, they discovered that their design increased the amount of power delivered. "We also found that the on-off, switch-like behavior of the diodes actually amplifies the power delivered, rather than reducing it, as previously thought," said Thibado. "The rate of change in resistance provided by the diodes adds an extra factor to the power."
> The team used a relatively new field of physics to prove the diodes increased the circuit's power. "In proving this power enhancement, we drew from the emergent field of stochastic thermodynamics and extended the nearly century-old, celebrated theory of Nyquist," said coauthor Pradeep Kumar, associate professor of physics and coauthor.
> According to Kumar, the graphene and circuit share a symbiotic relationship. Though the thermal environment is performing work on the load resistor, the graphene and circuit are at the same temperature and heat does not flow between the two.
> That's an important distinction, said Thibado, because a temperature difference between the graphene and circuit, in a circuit producing power, would contradict the second law of thermodynamics. "This means that the second law of thermodynamics is not violated, nor is there any need to argue that 'Maxwell's Demon' is separating hot and cold electrons," Thibado said.
I don't understand. It sounds like they're drawing useful work (I think?) from a system that's already at thermodynamic equilibrium. Doesn't that violate the 2nd law of thermodynamics?
I'm not sure that I understand either. From the abstract (which phys.org failed to link to):
> The system reaches thermal equilibrium and the rates of heat, work, and entropy production tend quickly to zero. However, there is power generated by graphene which is equal to the power dissipated by the load resistor.
Looks like the article is also on ArXiV: https://arxiv.org/abs/2002.09947
https://scholar.google.com/scholar?oi=bibs&hl=en&cluster=103...
Is it really a closed system at equilibrium?
Mozilla shuts project Iodide: Datascience documents in browsers
I did this! I killed it and I didn't mean to.
Ten (10) days ago, I filed an issue in the iodide project: "Compatibility with 'percent' notebook format" https://github.com/iodide-project/iodide/issues/2942
And then six (6) days ago, I added this comment to that issue: https://github.com/iodide-project/iodide/issues/2942#issueco...
And now it's almost dead, and I didn't mean to kill it.
But I also suggested that it would be great if conda-forge had a WASM build target:
- "Consider moving CPython patches upstream" https://github.com/iodide-project/pyodide/issues/635#issueco...
For students, being able to go to a URL and have a notebook interface with the SciPy stack preinstalled without needing to have an organization manage shell accounts and/or e.g. JupyterHub for every student should be worth the necessary budget allocation. Their local machines have plenty of CPU, storage, and memory for all but big data workloads.
Iodide is/was really cool. Pyiodide (much of the SciPy stack compiled to WASM) is also a great idea.
Jyve with latest JupyterLab, nbgrader, and configurable cloud storage could also solve.
Google Colab is an alternative to consider if you need to share Jupyter notebooks: https://colab.research.google.com/notebooks/intro.ipynb.
There are many ways to share reproducible Jupyter notebooks.
Google Colab now supports ipywidgets (js) in notebooks. While you can install additional packages in Colab, additional packages must be installed by each user (e.g. with `! pip install sympy` in an initial input cell) for each new kernel.
repo2docker builds a docker image from software dependency versions specified in e.g. requirements.txt, environment.yml, and/or a postInstall script and then installs a current version of JupyterLab in the container. Zero-to-BinderHub describes how to get BinderHub (which builds and launches containers) running on a hosting provider w/ k8s. awesome-python-in-education/blob/master/README.md#jupyter
Google AI Platform Notebooks is hosted JupyterLab.
awesome-jupyter > Hosted Notebook Solutions lists a number of services: https://github.com/markusschanta/awesome-jupyter#hosted-note...
awesome-python-in-education > Jupyter links to many Jupyter resources like nbgrader and BinderHub but not yet Jyve: https://github.com/quobit/awesome-python-in-education#jupyte...
Ask HN: What are good life skills for people to learn?
My initial thoughts; learn to drive, first aid, a sport, play an instrument, a language, how to manage finances, to speak in front of people.
- "Consumer science (a.k.a. home economics) as a college major" https://news.ycombinator.com/item?id=17894550
In no particular order:
- Food science; Nutrition
- Family planning: https://en.wikipedia.org/wiki/Family_planning
- Personal finance (see the link above for resources)
- How to learn
- How to teach [reading and writing, STEM, respect, compassion]
- Compassion for others' suffering
- How to considerately escape from unhealthy situations
- Coping strategies: https://en.wikipedia.org/wiki/Coping
- Defense mechanisms: https://en.wikipedia.org/wiki/Defence_mechanism
- Prioritization; productivity
- Goal setting; n-year planning; strategic alignment
Life skills: https://en.wikipedia.org/wiki/Life_skills
Khan Academy > Life Skills: https://www.khanacademy.org/college-careers-more
Four Keys Project metrics for DevOps team performance
> […] four key metrics that indicate the performance of a software development team:
> Deployment Frequency - How often an organization successfully releases to production
> Lead Time for Changes - The amount of time it takes a commit to get into production
> Change Failure Rate - The percentage of deployments causing a failure in production
> Time to Restore Service - How long it takes an organization to recover from a failure in production
Ask HN: Resources to encourage teen on becoming computer engineer?
Howdy HN
A teenager I am close with would like to become a computer engineer. Whet resources, books, podcasts, camps, or experiences do you recommend to support this teen's endeavor?
"Ask HN: Something like Khan Academy but full curriculum for grade schoolers?" [through undergrads] https://news.ycombinator.com/item?id=23794001
"Ask HN: How to introduce someone to programming concepts during 12-hour drive?" https://news.ycombinator.com/item?id=15454071
"Ask HN: Any detailed explanation of computer science" https://news.ycombinator.com/item?id=15270458 : topologically-sorted? Information Theory and Constructor Theory are probably at the top:
> A bottom-up (topologically sorted) computer science curriculum (a depth-first traversal of a Thing graph) ontology would be a great teaching resource.
> One could start with e.g. "Outline of Computer Science", add concept dependency edges, and then topologically (and alphabetically or chronologically) sort.
> https://en.wikipedia.org/wiki/Outline_of_computer_science
> There are many potential starting points and traversals toward specialization for such a curriculum graph of schema:Things/skos:Concepts with URIs.
> How to handle classical computation as a "collapsed" subset of quantum computation? Maybe Constructor Theory?
> https://en.wikipedia.org/wiki/Constructor_theory
https://westurner.github.io/hnlog/ ... Ctrl-F "interview", "curriculum"
CadQuery: A Python parametric CAD scripting framework based on OCCT
I wish more people would try this over openscad. I like openscad but the UI and with cadquery you use python! and now I noticed "CadQuery supports Jupyter notebook out of the box"
The jupyter-cadquery extension renders models with three.js via pythreejs in a sidebar with jupyterlab-sidecar: https://github.com/bernhard-42/jupyter-cadquery#b-using-a-do...
https://github.com/bernhard-42/jupyter-cadquery/blob/master/...
Array Programming with NumPy
Looks like there's a new citation for NumPy in town.
"Citing packages in the SciPy ecosystem" lists the existing citations for SciPy, NumPy, scikits, and other -Py things: https://www.scipy.org/citing.html ( source: https://github.com/scipy/scipy.org/blob/master/www/citing.rs... )
A better way to cite requisite software might involve referencing a https://schema.org/SoftwareApplication record in JSON-LD, RDFa, or Microdata; for example: https://news.ycombinator.com/item?id=24489651
But there's as of yet no way to publish JSON-LD, RDFa, or Microdata Linked Data from LaTeX with Computer Modern.
For some reason this struck me as inappropriate for the outlet. It's a nice piece as an introduction to array programming with numpy, but seemed out of place to me.
If, going forward, 5% of all papers that use NumPy to get their results actually cite this paper, it will be one of Nature's most cited papers every year.
There's an interesting trend of what content gets published in peer-reviewed journals vs. blogs/github/etc. I suspect there is an audience segment that strongly values peer reviewed pieces that are equivalent content wise to introductory material in a variety of formats.
I wonder if github should add a "Review" feature to provide a similar content authoring experience.
It would be nice if citing repositories were easier-- either for generating a reference for my own code or acknowledging when I've used someone else's code in my research.
There's tons of math and physics blogs that contain useful results that the author wanted to make available but didn't manage to incorporate into a paper. I wonder if there'd be any interest in a sort of GitHub for proofs? It could even use git, since (assuming consistency) isn't math just a DAG anyways (and therefore isomorphic to a neural net, as are all things).
You can get a free DOI for and archive a tag of a Git repo with FigShare or Zenodo.
If you have repo2docker REES dependency scripts (requirements.txt, environment.yml, postInstall,) in your repo, a BinderHub like https://mybinder.org can build and cache a container image and launch a (free) instance in a k8s cloud.
Journals haven't yet integrated with BinderHub.
Putting the suggested citation and DOI URI/URL in your README and cataloging citations in an e.g. wiki page may increase the crucial frequency of citation.
A Linked Data format for presenting well-formed arguments with #StructuredPremises would help to realize the potential of the web as a graph of resources which may satisfy formal inclusion criteria for #LinkedMetaAnalyses.
The issue is that none of the citation count engines (Google scholar, scopus, Web of Science...) count citations on those DOIs. So for a researcher who needs to somehow demonstrate impact through citation counts, it does not really help unfortunately.
We could reason about sites that index https://schema.org/ScholarlyArticle according to our own and others' observations. Google Scholar, Semantic Scholar, and Meta all index Scholarly Articles: they copy the bibliographic metadata and the abstract for archival and schoarly purposes.
AFAIU, e.g. Zotero and Mendeley do not crawl and index articles or attempt to parse bibliographic citations from the astounding plethora of citation styles [citationstyles, citationstyles_stylerepo] into a citation graph suitable for representative metrics [zenodo_newmetrics].
bitcoin.org/bitcoin.pdf does not have a DOI, does not have an ORCID [orcid], and is not published in any journal but is indexed by e.g. Google Scholar; though there are apparently multiple records referring to a ScholarlyArticle with the same name and author. Something like "Hell's Angels" (1930)? No DOI, no ORCID, no parseable PDF structure: not indexed.
AFAIU, Google Scholar does not yet index ScholarlyArticle (or SoftwareApplication < CreativeWork) bibliographic metadata. GScholar indexes an older set of bibliographic metadata from HTML <meta> tags and also attempts to parse PDFs. [gscholar_inclusion]
Google Scholar is also not (yet?) integrated with Google Dataset Search (which indexes https://schema.org/Dataset metadata).
FigShare DOIs and Zenodo DOIs are DataCite DOIs [figshare_howtocite, zenodo_principles]; which apparently aren't (yet?) all indexed by Google Scholar [rescience_gscholar].
IIUC, all papers uploaded to https://arxiv.org are indexed by Google Scholar. In order for arxiv-vanity.org [arxiv_vanity] to render a mobile-ready, font-resizeable HTML5 version of a paper uploaded to ArXiV, the PostScript source must be uploaded. Arxiv hosts certain categories of ScholarlyArticles.
JOSS (Journal of Open Source Software) has managed to get articles indexed by Google Scholar [rescience_gscholar]. They publish their costs [joss_costs]: $275 Crossref membership, DOIs: $1/paper:
> Assuming a publication rate of 200 papers per year this works out at ~$4.75 per paper
[citationstyles]: https://citationstyles.org
[citationstyles_stylerepo]: https://github.com/citation-style-language/styles
[gscholar_inclusion]: https://scholar.google.com/intl/en/scholar/inclusion.html#in...
[figshare_howtocite]: https://knowledge.figshare.com/articles/item/how-to-share-ci...
[zenodo_principles]: https://about.zenodo.org/principles/
[zenodo_newmetrics]: https://www.frontiersin.org/articles/10.3389/frma.2017.00013...
[rescience_gscholar]: https://github.com/ReScience/ReScience/issues/38
[arxiv_vanity]: https://www.arxiv-vanity.com/
[joss_costs]: https://joss.theoj.org/about#costs
[orcid]: https://en.wikipedia.org/wiki/ORCID
Do you like the browser bookmark manager?
How do you think it compares to services like webcull.com, raindrop.io, or getpocket.com? Have they advanced the field to the point that it's worth switching?
Things I'd add to browser bookmark managers someday:
- Support for (persisting) bookmarks tags. From the post re: the re-launch of del.icio.us: https://news.ycombinator.com/item?id=23985623
> "Allow reading and writing bookmark tags" https://bugzilla.mozilla.org/show_bug.cgi?id=1225916
> Notes re: how this could be standardized with JSON-LD: https://bugzilla.mozilla.org/show_bug.cgi?id=1225916#c116
> The existing Web Experiment for persisting bookmark tags: https://github.com/azappella/webextension-experiment-tags/bl...
- Standard search features like operators: ((term) AND (term2)) OR term3
- Regex search
- (Chrome) show the createdDate and allow (non-destructive) sort by date
- Native sync API for syncing to zero or more bookmarks / personal data storage providers
- Support for integration with extensions that support actual resource metadata like Zotero
- Linked Data support: extract and store bibliographic metadata like Zotero and OpenLink Structured Data Sniffer
What are the current limitations of the WebExtensions Bookmarks API (now supported by Firefox, Chrome, Edge, and hopefully eventually Safari)?: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/Web...
NIST Samate – Source Code Security Analyzers
Additional lists of static analysis, dynamic analysis, SAST, DAST, and other source code analysis tools:
OWAP > Source Code Analysis Tools: https://owasp.org/www-community/Source_Code_Analysis_Tools
https://analysis-tools.dev/ (supports upvotes and downvotes)
analysis-tools-dev/static-analysis: https://github.com/analysis-tools-dev/static-analysis
analysis-tools-dev/dynamic-analysis: https://github.com/analysis-tools-dev/dynamic-analysis
devsecops/awesome-devsecops: https://github.com/devsecops/awesome-devsecops , https://github.com/TaptuIT/awesome-devsecops
kai5263499/awesome-container-security: https://github.com/kai5263499/awesome-container-security
https://en.wikipedia.org/wiki/DevOps#DevSecOps,_Shifting_Sec... :
> DevSecOps is an augmentation of DevOps to allow for security practices to be integrated into the DevOps approach. The traditional centralised security team model must adopt a federated model allowing each delivery team the ability to factor in the correct security controls into their DevOps practices.
awesome-safety-critical: https://awesome-safety-critical.readthedocs.io/en/latest/
A Handwritten Math Parser in 100 lines of Python
Slightly more amusingly, but non-equivalently:
import sys
from functools import reduce
from operator import add, sub, mul, truediv as div
def calc(s, ops=[(add, '+'), (sub, '-'), (mul, '*'), (div, '/')]):
if not ops:
return float(s)
(func, token), *remaining_ops = ops
return reduce(func, (calc(part, remaining_ops) for part in s.split(token)))
if __name__ == '__main__':
print(calc(''.join(sys.argv[1:])))
This is pretty clever. There's a video by Jonathan Blow, creator of Braid and currently making a programming language called Jai, that talks about parsing expressions with order of operations. He gives an simple approach that apparently was discovered recently (though he didn't remember the paper name) that does this in a single pass and hardly any more complicated than what you wrote. You can see it here [1]. He also trashes on using yacc/bison before that and on most academic parsing theory.
Reverse Polish notation (RPN) > Converting from infix notation https://en.wikipedia.org/wiki/Reverse_Polish_notation#Conver... > Shunting-yard algorithm https://en.wikipedia.org/wiki/Shunting-yard_algorithm
Infix notation supports parentheses.
Infix notation: 3 + 4 × (2 − 1)
RPN: 3 4 2 1 − × +
PEP – An open source PDF editor for Mac
I have a dream... that one day people will name their projects with names that don't exist on google yet.
If you search for PEP now you'll find python enhancement proposals, and the "Philippine Entertainment Portal" and the stock code for PepsiCo.
I wish that once people do name their project, they would assign it a 128-bit random number in lower case hex, and include that number on any web page that they would like people searching for their project to find.
That way once I know that say PEP the PDF editor exists and find its 128-bit number (let's say that is 379dd864b16eaca3ce94c15a6bdfcc73), at least I can subsequently toss a +379dd864b16eaca3ce94c15a6bdfcc73 on my searches to effectively let the search engine know I want PEP the PDF editor results rather than PEP the python enhancement results or PEP the entertainment portal results or PEP that refreshing beverage company stock symbol.
"xxd -l 16 -p /dev/urandom" is a handy way to get a 128-bit random hex number. A UUID generator works, too, although they usually include some punctuation you will need to delete and you might have to lower case their output.
> I wish that once people do name their project, they would assign it a 128-bit random number in lower case hex
We already have something similar: URLs.
tzs is proposing URNs rather than textual program names. A URL is unnecessarily specific (though I suppose you could anycast URL resolution)
> RFC 4122 defines a Uniform Resource Name (URN) namespace for UUIDs. A UUID presented as a URN appears as follows:[1]
> > urn:uuid:123e4567-e89b-12d3-a456-426655440000
https://en.wikipedia.org/wiki/Universally_unique_identifier#...
Version 4 UUIDs have 122 random bits (out of 128 bits total).
In Python:
>>> import uuid
>>> _id = uuid.uuid4()
>>> _id.urn
'urn:uuid:4c466878-a81b-4f22-a112-c704655fa4ee'
>>> _id.hex
'4c466878a81b4f22a112c704655fa4ee'
{"@context": "https://schema.org",
"@type": "WebPage",
"about": {
"@type": "SoftwareApplication",
"identifier": "urn:uuid:4c466878-a81b-4f22-a112-c704655fa4ee",
"url": ["", ""],
"name": [
"a schema.org/SoftwareApplication < CreativeWork < Thing",
{"@value": "a rose by any other name",
"@language": "en"}]}}
<body vocab="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" typeof="WebPage">
<div property="about" typeof="SoftwareApplication">
<meta property="identifier" content="urn:uuid:4c466878-a81b-4f22-a112-c704655fa4ee"/>
<span property="name">a schema.org/SoftwareApplication < CreativeWork < Thing</span>
<span property="name" lang="en">a rose by any other name</span>
</div>
</body>
<div itemtype="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" itemscope>
<link itemprop="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" href="https://schema.org/" />
<div itemprop="about" itemtype="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" itemscope>
<meta itemprop="identifier" content="urn:uuid:4c466878-a81b-4f22-a112-c704655fa4ee" />
<meta itemprop="name" content="a <a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">schema.org/SoftwareApplication < CreativeWork < Thing"/>
<meta itemprop="name" content="a rose by any other name" lang="en"/>
</div>
</div>The Unix timestamp will begin with 16 this Sunday
Redox: Unix-Like Operating System in Rust
I'm Jeremy Soller, the creator of Redox OS. Let me know if you have questions!
Are there tools to support static analysis and formal methods in Rust yet?
From https://news.ycombinator.com/item?id=21839514 re: awesome-safety-critical https://awesome-safety-critical.readthedocs.io/en/latest/ :
> > Does Rust have a chance in mission-critical software? (currently Ada and proven C niches) https://www.reddit.com/r/rust/comments/5iv5j7/does_rust_have...
FWIU, Sealed Rust is in progress.
And there's also RustPython for the userspace.
Ask HN: How are online communities established?
HN, Reddit, Stack Overflow, etc. are all established communities with users. How do you start a community when you don't have any users?
You should read the book People Powered by Jono Bacon. Has some really good insights and is a 101 course on exactly this.
Seconded. "People Powered: How Communities Can Supercharge Your Business, Brand, and Teams" (2019) https://g.co/kgs/CF5TEk
"The Art of Community: Building the New Age of Participation" (2012) https://g.co/kgs/P2V1kn
"Tribes: We need you to lead us" (2011) https://g.co/kgs/T8jaFS
The 1% 'rule' https://en.wikipedia.org/wiki/1%25_rule_(Internet_culture) :
> In Internet culture, the 1% rule is a rule of thumb pertaining to participation in an internet community, stating that only 1% of the users of a website add content, while the other 99% of the participants only lurk. Variants include the 1–9–90 rule (sometimes 90–9–1 principle or the 89:10:1 ratio),[1] which states that in a collaborative website such as a wiki, 90% of the participants of a community only consume content, 9% of the participants change or update content, and 1% of the participants add content.
... Relevant metrics:
- Marginal cost of service https://en.wikipedia.org/wiki/Marginal_cost
- Customer acquisition cost: https://en.wikipedia.org/wiki/Customer_acquisition_cost
- [Quantifiable and non-quantifiable] Customer Lifetime Value: https://en.wikipedia.org/wiki/Customer_lifetime_value
Last words of the almost-cliche community organizer surrounded by dormant accounts: "Network effects will result in sufficient (grant) funding"
Business model examples that may be useful for building and supporting sustainable communities with clear Missions, Objectives, and Criteria for Success: https://gist.github.com/ndarville/4295324
Python Documentation Using Sphinx
I usually generate new Python projects with a cookiecutter; such as cookiecutter-pypackage. I like the way that cookiecutter-pypackage includes a Makefile which has a `docs` task so that I can call `make docs` to build the sphinx docs in the docs/ directory which include:
- a /docs/readme.rst that includes the /README.rst as the first document in the toctree
- a sensible set of default documents: readme (.. include:: /README.rst), installation, usage, modules (sphinx-autodoc output), contributing, authors, history (.. include:: /HISTORY.rst)
- a sphinx conf.py that sets the docs' version and release attributes to pkgname.__version__; so that the version number only needs to be changed in one place (as long as setup.py or setup.cfg also read the version string from pkgname.__version__)
- a default set of extensions: ['sphinx.ext.autodoc', 'sphinx.ext.viewcode'] that generates API docs and includes '[source]' hyperlinks from the generated API docs to the transcluded syntax-highlighted source code and links back to the API docs from the source code
https://github.com/audreyfeldroy/cookiecutter-pypackage/tree...
There are a few styles of docstrings that Sphinx can parse and include in docs with e.g. sphinx-autodoc:
`:param, :type, :returns, :rtype` docstrings (which OP uses; and which pycontracts can read runtime parameter and return type contracts from https://andreacensi.github.io/contracts/ (though Python 3 annotations are now the preferred style for compile or editing-time typechecks))
Numpydoc docstrings: https://numpydoc.readthedocs.io/en/latest/format.html
Googledoc docstrings: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/
You can use Markdown with Sphinx in at least three ways:
MyST Markdown supports Sphinx and Docutils roles and directives. Jupyter Book builds upon MyST Markdown. With Jupyter Book, you can include Jupyter notebooks (which can include MyST Markdown) in your Sphinx docs. Executable notebooks are a much easier way to include up-to-date code outputs in docs. https://myst-parser.readthedocs.io/en/latest/
Sphinx (& ReadTheDocs) w/ recommonmark: https://docs.readthedocs.io/en/stable/intro/getting-started-...
Nbsphinx predates Jupyter Book and doesn't yet support MyST Markdown, but does support Markdown cells in Jupyter notebooks. Nbsphinx includes a parser for including .ipynb Jupyter notebooks in Sphinx docs. nbsphinx supports raw RST (ReST) cells in Jupyter notebooks and has great docs: https://nbsphinx.readthedocs.io/en/latest/
Nbdev is another approach; though it's not Sphinx:
> nbdev is a library that allows you to fully develop a library in Jupyter Notebooks, putting all your code, tests and documentation in one place.
> [...] Add %nbdev_export flags to the cells that define the functions you want to include in your python modules
https://github.com/fastai/nbdev
A few additional sources of docs for Sphinx and ReStructuredText:
Read The Docs docs > Getting Started with Sphinx > External Resources https://docs.readthedocs.io/en/stable/intro/getting-started-...
CPython Devguide > "Documenting Python" https://devguide.python.org/documenting/
"How to write [Linux] kernel documentation" https://www.kernel.org/doc/html/latest/doc-guide/index.html
awesome-sphinxdoc: https://github.com/yoloseem/awesome-sphinxdoc
... "Ask HN: Recommendations for Books on Writing [for engineers]?" https://news.ycombinator.com/item?id=23945580
Traits of good remote leaders
And the evidence for these considerations is what?
They reference a study: https://link.springer.com/article/10.1007%2Fs10869-020-09698..., titled: "Who Emerges into Virtual Team Leadership Roles? The Role of Achievement and Ascription Antecedents for Leadership Emergence Across the Virtuality Spectrum".
Fortunately the references are free to view.
"Table 4 – Correlation of Development Phases, Coping Stages and Comfort Zone transitions and the Performance Model" in "From Comfort Zone to Performance Management" White (2008) tabularly correlates the Tuckman group development phases (Forming, Storming, Norming, Performing, Adjourning) with the Carnall coping cycle (Denial, Defense, Discarding, Adaptation, Internalization) and Comfort Zone Theory (First Performance Level, Transition Zone, Second Performance Level), and the White-Fairhurst TPR model (Transforming, Performing, Reforming). The ScholarlyArticle also suggests management styles for each stage (Commanding, Cooperative, Motivational, Directive, Collaborative); and suggests that team performance is described by chained power curves of re-progression through these stages.
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C43&q=%E2...
IDK what's different about online teams in regards to performance management?
Show HN: Eiten – open-source tool for portfolio optimization
Is it possible to factor (e.g. GRI) sustainability criteria into the portfolio fitness function? https://news.ycombinator.com/item?id=21922558
My concern is that - like any other portfolio optimization algorithm - blindly optimizing on fundamentals and short term returns will lead to investing in firms who just dump external costs onto people in the present and future; so, screening with sustainability criteria is important to me.
From https://news.ycombinator.com/item?id=19111911 :
> awesome-quant lists a bunch of other tools for algos and superalgos: https://github.com/wilsonfreitas/awesome-quant
This is perfect. Thank you for sharing this.
I might start implementing some of these but would love for someone else to add a few PRs as well. The code is pretty modular especially if we want to add new strategies.
(Sustainable) Index ETFs in the stocks.txt universe would likely be less sensitive to single performers' effects in unbalanced portfolios.
> pyfolio.tears.create_interesting_times_tear_sheet measures algorithmic trading algorithm performance during "stress events" https://github.com/quantopian/pyfolio/blob/03568e0f328783a6a...
Ask HN: Any well funded tech companies tackling big, meaningful problems?
Are there any well funded tech startups / companies tackling major societal problems? Any of these fair game: https://en.wikipedia.org/wiki/List_of_global_issues
----
I don't see or hear of any and want to know if this is just my bias or if there really is a shortage of resources in tech being allocated to solving the worlds most important problems. I'm sure I'm not the only engineer that's looking out for companies like this.
Ran into this previous Ask HN (https://news.ycombinator.com/item?id=24168902) that asked a similar question. However, here I wanna focus on the better funded efforts (not side projects, philanthropy etc).
One example I've heard so far is Tesla. Any others?
You can make an impact by solving important local and global problems by investing your time, career, and savings; by listing and comparing solutions.
As a labor market participant, you can choose to work for places that have an organizational mission that strategically aligns with local, domestic, and international objectives.
https://en.wikipedia.org/wiki/Strategic_alignment ... "Schema.org: Mission, Project, Goal, Objective, Task" https://news.ycombinator.com/item?id=12525141
As an investor, you can choose to invest in organizations that are making the sort of impact you're looking for: you can impact invest.
https://en.wikipedia.org/wiki/Impact_investing
You mentioned "List of global issues"; which didn't yet have a link to the UN Sustainable Development Goals (the #GlobalGoals). I just added this to the linked article:
> As part of the 2030 Agenda for Sustainable Development, the UN Millenium Development Goals (2000-2015) were superseded by the UN Sustainable Development Goals (2016-2030), which are also known as The Global Goals. There are associated Targets and Indicators for each Global Goal.
There are 17 Global Goals.
Sustainability reporting standards can align with the Sustainable Development Goals. For example, the GRI standards are now aligned with the UN Sustainable Development Goals.
https://en.wikipedia.org/wiki/Sustainable_Development_Goals
Investors, fund managers, and potential employees can identify companies which are making an impact by reviewing corporate sustainability and ESG reports.
From https://www.undp.org/content/undp/en/home/sustainable-develo... :
> SDG Target 12.6: "Encourage companies, especially large and transnational companies, to adopt sustainable practices and to integrate sustainability information into their reporting cycle"
From https://news.ycombinator.com/item?id=21302926 :
> > What are some of the corporate sustainability reporting standards?
> > From https://en.wikipedia.org/wiki/Sustainability_reporting#Initi... :
> >> Organizations can improve their sustainability performance by measuring (EthicalQuote (CEQ)), monitoring and reporting on it, helping them have a positive impact on society, the economy, and a sustainable future. The key drivers for the quality of sustainability reports are the guidelines of the Global Reporting Initiative (GRI),[3] (ACCA) award schemes or rankings. The GRI Sustainability Reporting Guidelines enable all organizations worldwide to assess their sustainability performance and disclose the results in a similar way to financial reporting.[4] The largest database of corporate sustainability reports can be found on the website of the United Nations Global Compact initiative.
> >The GRI (Global Reporting Initiative) Standards are now aligned with the UN Sustainable Development Goals (#GlobalGoals). https://en.wikipedia.org/wiki/Global_Reporting_Initiative
> >> In 2017, 63 percent of the largest 100 companies (N100), and 75 percent of the Global Fortune 250 (G250) reported applying the GRI reporting framework.[3]
What are some good ways to search for companies who (1) do sustainability reports, (2) engage in strategic alignment in corporate planning sessions, (3) make sustainability a front-and-center issue in their company's internal and external communications?
What are some examples of companies who have a focus on sustainability and/or who have developed a nonprofit organization for philanthropic missions which are sometimes best accounted for as a distinct organization or a business unit (which can accept and offer receipts for donations as a non-profit)?
How can an employee drive change in a small or a large company? Identify opportunities to deliver value and goodwill. Read through the Global Goals, Targets, and Indicators; and get into the habit of writing down problems and solutions.
3 pillars of [Corporate] Sustainability: (Environment (Society (Economy))). https://en.wikipedia.org/wiki/Sustainability#Three_dimension...
"Launch HN: Charityvest (YC S20) – Employee charitable funds and gift matching" https://news.ycombinator.com/item?id=23907902 :
> We created a modern, simple, and affordable way for companies to include charitable giving in their suite of employee benefits.
> We give employees their own tax-deductible charitable giving fund, like an “HSA for Charity.” They can make contributions into their fund and, from their fund, support any of the 1.4M charities in the US, all on one tax receipt.
> Using the funds, we enable companies to operate gift matching programs that run on autopilot. Each donation to a charity from an employee is matched automatically by the company in our system.
> A company can set up a matching gift program and launch giving funds to employees in about 10 minutes of work.
"Salesforce Sustainability Cloud Becomes Generally Available" https://news.ycombinator.com/item?id=22068522 :
> Are there similar services for Sustainability Reporting and accountability?
Column Names as Contracts
I really like the idea of being more thoughtful about naming columns and being more explicit about the “type” of data contained in them.
Is this idea already known among data modelers or data engineers?
I’d love to read any other references, if available.
I am a data architect in my day job. Within the realm of data management, I'd say "metadata management" [1] is the general category this fits within.
I would say, yes this idea is known/very common, as data architecture is as much about the descriptive language we use as anything. I mean, "business glossaries", taxonomy, even just naming conventions [2] in coding, these are all related.
If you build enough databases/tables or even code yourself, you inevitably come across the "how to name things" problem [3]. If all you have to sort on for the known meaning of a thing (column, table, file, etc.) is a single string value, then encoding meaning into it is quite common. This way, a sort creates a kind of "grouping". Many database vendors follow standard naming conventions - such as Oracle, for example [4]. It is considered a best practice when designing/building the metadata for a large system, to establish a naming convention. Among other things, it makes finding things easier, as well as all the potential for automation.
You get all kinds of variations on this, such as, should the "ID_" come as a prefix or a suffix (i.e. "_ID"). One's initial thought is to use it as a prefix so all the related types group together, but then that becomes much more difficult if you want to sort items by their functional area (e.g. DRIVER_ID, DRIVER_IND, etc.).
One other place you see something similar is in "smart numbers" which is an eternal argument - should I use a "dumb identifier" (GUID, integer) or a "smart one" (one encoding additional meaning) [5].
I mean, basically, any time you can encode information in the meta-data of data, I think you can then operate on it by following "convention over configuration" (as mentioned elsewhere in the discussion comments).
The only problem I see is that such conventions can, at times be limiting - depending on the length of your metadata columns, and the variability you are trying to capture - which is why I believe, generally, metadata is often better separated and linked to the data it describes - this decoupling allows for much more descriptive metadata than one could encode in simple a single string value. Certainly, you can get a long way with an approach like this, but I suspect you would run into 80/20 rule limitations.
Using naming in this way is a form of tight coupling, which could be seen as an anti-pattern in terms of meta-data flexibility, in some cases.
[1] https://en.wikipedia.org/wiki/Metadata_management
[2] https://en.wikipedia.org/wiki/Naming_convention_(programming...
[3] https://martinfowler.com/bliki/TwoHardThings.html
[4] https://oracle-base.com/articles/misc/naming-conventions
In terms of database normalization, delimiting multiple fields within a column name field violates the "atomic columns" requirement of the first though sixth normal forms (1NF - 6NF)
https://en.wikipedia.org/wiki/Database_normalization
Are there standards for storing columnar metadata (that is, metadata about the columns; or column-level metadata)?
In terms of columns, SQL has (implicit ordinal, name, type) and then primary key, index, and [foreign key] constraints.
RDFS (RDF Schema) is an open W3C linked data standard. An rdf:Property may have a rdfs:domain and a rdfs:range; where the possible datatypes are listed as instances of rdfs:range. Primitive datatypes are often drawn from XSD (XML Schema Definition), or https://schema.org/ . An rdfs:Class instance may be within the rdfs:domain and/or the rdfs:range of an rdf:Property.
RDFS is generally not sufficient for data validation; there are a number of standards which build upon RDFS: W3C SHACL (Shapes and Constraint Language), W3C CSVW (CSV on the Web).
There is some existing work on merging JSON Schema and SHACL.
CSVW builds upon the W3C "Model for Tabular Data and Metadata on the Web"; which supports arbitrary "annotations" on columns. CSVW can be represented as any RDF representation: Turtle/Trig/M3, RDF/XML, JSON-LD.
https://www.w3.org/TR/tabular-data-primer/
https://www.w3.org/TR/tabular-data-model/ :
> an annotated tabular data model: a model for tables that are annotated with metadata. Annotations provide information about the cells, rows, columns, tables, and groups of tables […]
...
From https://twitter.com/westurner/status/901992073846456321 :
> "7 metadata header rows (column label, property URI path, DataType, unit, accuracy, precision, significant figures)" https://wrdrd.github.io/docs/consulting/linkedreproducibilit...
...
From https://twitter.com/westurner/status/1295774405923147778 :
> Relevant: https://discuss.ossdata.org/ topics: "Linked Data formats, tools, challenges, opportunities; CSVW, https://schema.org/Dataset , https://schema.org/ScholarlyArticle " https://discuss.ossdata.org/t/linked-data-formats-tools-chal...
> "A dataframe protocol for the PyData ecosystem" https://discuss.ossdata.org/t/a-dataframe-protocol-for-the-p...
> A .meta protocol should implement the W3C Tabular Data Model: [...]
...
The various methods of doing CSV2RDF and R2RML (SQL / RDB to RDF Mapping) each have a way to specify additional metadata annotations. None stuff data into a column name (which I'm also guilty of doing with e.g. "columnspecs" in a small line-parsing utility called pyline that can cast columns to Python types and output JSON lines).
...
Even JSON5 is insufficient when it comes to representing e.g. complex fractions: there must be a tbox (schema) in order to read the data out of the abox (assertions; e.g. JSON). JSON-LD is sufficient for representation; and there are also specs like RDFS, SHACL, and CSVW.
Abox: https://en.wikipedia.org/wiki/Abox
I see the line of thinking you're going down. There are ISO standards for data types, in a sense I could see why one would seek a standard language for defining the metadata/specification of a type as data. Have to really think about that some more.. in a way a regex could be seen as a compact form of expressing the capability of a column in terms of value ranges or domains, but to define the meaning of the data, not so much.
Your interpretation of the atomic columns requirement is a little different than my understanding. That requirement of normalization only applies to the "cells" of columnar data, it says nothing about encoding meaning into column names, which are themselves simply descriptive metadata.
I mean, for sure you wouldn't want to encode many values/meanings into a column name (some systems have length restrictions that would make that impossible, I'm not sure it makes sense anyway), but just pointing out that technically the spec does not make that illegal. Certainly, adding minor annotations within the name of a column separated by a supported delimiter does not, in my opinion, violate normalization rules at all. I mean things like "ID_" or similar.
Have you looked at INFORMATION_SCHEMA in SQL databases? [1] You mentioned SQL metadata and constraints, that is as close to a standard feature for querying that information there is, some databases do it using similar but non-standard ways (Oracle for example).
Also, not standard but, many relational databases support extended properties or Metadata for objects (tables, views, columns, etc.) - you can often come up with your own scheme although rarely do I see people utilize these features. [2] [3]
At some point it feels like we are more talking about type definitions and annotations, applied to data columns.
Maybe like, BNF [4] for purely data table columns (which are essentially types)?
[1] https://en.wikipedia.org/wiki/Information_schema
[2] http://www.postgresql.org/docs/current/static/sql-comment.ht...
[3] https://docs.microsoft.com/en-us/sql/relational-databases/sy...
Graph Representations for Higher-Order Logic and Theorem Proving (2019)
ONNX (and maybe RIF) are worth mentioning.
ONNX: https://onnx.ai/ :
> ONNX is an open format built to represent machine learning models. ONNX defines a common set of operators - the building blocks of machine learning and deep learning models - and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers
RIF (~FOL): https://en.wikipedia.org/wiki/Rule_Interchange_Format
Datalog (not Turing-complete): https://en.wikipedia.org/wiki/Datalog
HOList Benchmark: https://sites.google.com/view/holist/home
"HOList: An Environment for Machine Learning of Higher-Order Theorem Proving" (2019) https://arxiv.org/abs/1904.03241
> Abstract: We present an environment, benchmark, and deep learning driven automated theorem prover for higher-order logic. Higher-order interactive theorem provers enable the formalization of arbitrary mathematical theories and thereby present an interesting, open-ended challenge for deep learning. We provide an open-source framework based on the HOL Light theorem prover that can be used as a reinforcement learning environment. HOL Light comes with a broad coverage of basic mathematical theorems on calculus and the formal proof of the Kepler conjecture, from which we derive a challenging benchmark for automated reasoning. We also present a deep reinforcement learning driven automated theorem prover, DeepHOL, with strong initial results on this benchmark.
I wonder why they don't mention any work based on transformer architectures? The recent work on solving differential equations based on expressions in reverse polish notation seemed like a reasonable idea to apply to theorem proving as well.
Really cool work though!
A transformer is unable to really represent logic, let alone higher order logic and theorem proving.
A transformer is a universal function approximator. The question is whether it can do so reasonably efficiently. Trained on natural language linear sequences, I’m with you. Trained on abstract logical graph representations? I don’t think that question’s answered yet, unless I’m missing something.
How do transformers handle with truth tables, logical connectives, and propositional logic / rules of inference, and first-order logic?
Truth table: https://en.wikipedia.org/wiki/Truth_table
Logical connective: https://en.wikipedia.org/wiki/Logical_connective
Propositional logic: https://en.wikipedia.org/wiki/Propositional_calculus
Rules of inference: https://en.wikipedia.org/wiki/Rule_of_inference
DL: Description logic: https://en.wikipedia.org/wiki/Description_logic (... The OWL 2 profiles (EL, QR, RL; DL, Full) have established decideability and complexity: https://www.w3.org/TR/owl2-profiles/ )
FOL: First-order logic: https://en.wikipedia.org/wiki/First-order_logic
HOL: Higher-order logic: https://en.wikipedia.org/wiki/Higher-order_logic
In terms of regurgitating without critical reasoning?
Critical reasoning: https://en.wikipedia.org/wiki/Critical_thinking
Think about a program that can write code but not execute it. It’s not hard to get a transformer to learn to write python code like merge sort or simple arithmetic code, even though a transformer can’t reasonably learn to sort, nor can it learn simple arithmetic. It’s an important disambiguation. In one view it appears it can’t (learn to sort) and in another it demonstrably can (learn to code up a sort function). It can learn to usefully manipulate the language without needing the capacity to execute the language. They probably can’t learn to “execute” what you’re talking about (execute being a loose analogy), but I’d say the jury’s out on whether they can learn to usefully manipulate it.
Transformer is a function from seq of symbols to seq of symbols. For example, truth table is exactly similar kind of function from variables to [True, False] alphabet.
Transformer can represent some very complex logical operations, and per this article is turing complete: https://arxiv.org/abs/1901.03429, meaning any computable function, including theorem prover can be represented as transformer.
Another question is if it is feasible/viable/rational to build transformer for this? My intuition says: no.
Show HN: Linux sysadmin course, eight years on
Almost eight years ago I launched an online “Linux sysadmin course for newbies” here at HN.
It was a side-project that went well, but never generated enough money to allow me to fully commit to leaving the Day Job. After surviving the Big C, and getting made redundant I thought I might improve and relaunch it commercially – but my doctors are a pessimistic bunch, so it looked like I didn’t have the time.
Instead, I rejigged/relaunched it via a Reddit forum this February as free and open - and have now gathered a team of helpers to ensure that it keeps going each month even after I can’t be involved any longer.
It’s a month-long course which restarts each month, so “Day 1” of September is this coming Monday.
It would be great if you could pass the word on to anyone you know who may be the target market of those who: “...aspire to get Linux-related jobs in industry - junior Linux sysadmin, devops-related work and similar”.
[0] http://www.linuxupskillchallenge.org/
[1] https://www.reddit.com/r/linuxupskillchallenge/
[2] http://snori74.blogspot.com/2020/04/health-status.html
There are a number of resources that may be useful for your curriculum for this project listed in "Is there a program like codeacademy but for learning sysadmin?" https://news.ycombinator.com/item?id=19469266 :
> [ http://www.opsschool.org/ , https://github.com/kahun/awesome-sysadmin/blob/master/README... , https://github.com/stack72/ops-books , https://landing.google.com/sre/books/ , https://response.pagerduty.com/ (Incident Response training)]
To that I'd add that K3D (based on K3S, which is now a CNCF project) runs Kubernetes (k8s) in Docker containers. https://github.com/rancher/k3d
For zero-downtime (HA: High availability) deployments, "Zero-Downtime Deployments To a Docker Swarm Cluster" describes Rolling Updates and Blue-Green Deployments; with illustrations: https://github.com/vfarcic/vfarcic.github.io/blob/master/doc...
For git-push style deployment with more of a least privileges approach (which also has more moving parts) you could take a look at: https://github.com/dokku/dokku-scheduler-kubernetes#function...
And also reference ansible molecule and testinfra for writing sysadmin tests and the molecule vagrant driver for testing docker configurations. https://www.jeffgeerling.com/blog/2018/testing-your-ansible-...
https://molecule.readthedocs.io/en/latest/
https://testinfra.readthedocs.io/en/latest/ :
> With Testinfra you can write unit tests in Python to test actual state of your servers configured by management tools like Salt, Ansible, Puppet, Chef and so on.
> Testinfra aims to be a Serverspec equivalent in python and is written as a plugin to the powerful Pytest test engine.
I wasn't able to find a syllabus or a list of all of the daily posts? Are you focusing on DevOps and/or DevSecOps skills?
EDIT: The lessons are Markdown files in a Git repo: https://github.com/snori74/linuxupskillchallenge
Links to each lesson, the title and/or subjects of the lesson, and the associated reddit posts might be useful in a Table of Contents in the README.md.
Thanks, but most of that would be way over the top for my "newbies".
However, You must be the third or fourth person today to suggest that I add a TOC - so that is something I think I'll need to look at!
Maybe most useful as resources for further study.
Looks like Day 20 covers shell scripting. A few things worth mentioning:
You can write tests for shell scripts and write TAP (Test Anything Protocol) -formatted output: https://testanything.org/producers.html#shell
Quoting in shell scripts is something to be really careful about:
> This and this do different things:
# prints a newline
echo $(echo "-e a\nb")
# prints "-e a\nb"
echo "$(echo "-e a\nb")"
LearnXinYminutes has a good bash reference: https://learnxinyminutes.com/docs/bash/
And an okay Ansible reference, which (like Ops School) we should contribute to: https://learnxinyminutes.com/docs/ansible/
Why do so many pros avoid maintaining shell scripts and writing one-off commands that they'll never remember to run again later?
...
It may be helpful to format these as Jupyter notebooks with input and output cells.
- Ctrl-Shift-Minus splits a cell at the cursor
- M and Y toggle a cell between Markdown and code
If you don't want to prefix every code cell line with a '!' so that the ipykernel Jupyter python kernel (the default kernel) executes the line with $SHELL, you can instead install and select bash_kernel; though users attempting to run the notebooks interactively would then need to also have bash_kernel installed: https://github.com/takluyver/bash_kernel
You can save a notebook .ipynb to any of a number of Markdown and non-Markdown formats https://jupytext.readthedocs.io/en/latest/formats.html#markd... ; unfortunately jupytext only auto-saves to md without output cell content for now: https://github.com/mwouts/jupytext/issues/220
You can make reveal.js slides (that do include outputs) from a notebook: https://gist.github.com/mwouts/04a6dfa571bda5cc59fa1429d1309...
With nbconvert, you can manually save an .ipynb Jupyter notebook as Markdown which includes the cell outputs w/ File > "Download as / Export Notebook as" > "Export notebook to Markdown" or with the CLI: https://nbconvert.readthedocs.io/en/latest/usage.html#conver...
jupyter convert --to markdown
jupyter convert --help
jupyter-book build mybook/
From https://westurner.github.io/tools/#bash :
type bash
bash --help
help help
help type
apropos bash
info bash
man bash
man man
info info
info bash -n "Bash Startup Files"
...
Re: dotfiles, losing commands that should've been logged to HISTFILE when running multiple bash sessions and why I wrote usrlog.sh: https://westurner.github.io/hnlog/#comment-20671184 (Ctrl-F for: "dotfiles", "usrlog.sh", "inputrc")
https://github.com/webpro/awesome-dotfiles
...
awesome-sysadmin > resources: https://github.com/kahun/awesome-sysadmin#resources
Software supply chain security
Estimates of prevalence do assume detection. How would we detect that a dependency that was installed a few deployments and reboots ago was compromised?
How does the classic infosec triad (Confidentiality, Integrity, Availability) apply to software supply chain security?
Confidentiality: Presumably we're talking about open source projects; which aren't confidential. Projects may request responsible disclosure in an e.g. security.txt; and vuln reports may be confidential for at least a little while.
Integrity: Secure transport protocols, checksums, and cryptographic code signing are ways to mitigate data integrity risks. GitHub supports SSH, 2FA, and GPG keys. Can all keys in the package signature keyring be used to sign any package? Can we verify a public key over a different channel? When we specify exact versions of software dependencies, can we also record package hashes which the package installer(s) will verify?
Availability: What are the internal and external data, network, and service dependencies for the development and deployment DevSecOps workflows? Can we deploy from local package mirrors? Who is responsible for securing and updating local package mirrors? Are these service dependencies all HA? Does everything in this system also depend upon the load balancer? Does our container registry support e.g. Docker Notary (TUF)? How should we mirror TUF package repos?
See also: "Guidance for [[transparent] proxy cache] partial mirrors?" https://github.com/theupdateframework/specification/issues/1...
A toolset that answers some of your questions is grafeas- a metadata store at https://github.com/grafeas/grafeas- and kritis, a policy engine at https://github.com/grafeas/kritis.
Cheers.
Thanks for the links. Do you know how this toolset helps to mitigate/prevent what is called in the GitHub blogpost "Supply chain compromises". Quickly checked around and couldn't find anything that applies to the dependencies of applications/binaries before they land into the target runtime (i.e k8s).
Have you seen these preso slides
https://www.slideshare.net/mobile/aysylu/q-con-sp-software-s....
They walk through one of the workflows (end state is deploying to k8s).
Grafeas is a metadata store, Kritis is a policy engine that plugs into k8s as an admission controller- blessing the "admission" (running) of an image in a namespace.
There are existing tools for each language/runtime that produce known vuln lists for individual artifacts in the language ecosystem. These you feed into Grafeas. And you have your CI pipeline providing manifests for each of your built images that contain all upstream dependencies (these produced from each app's build tool). Then at deploy time, Kritis checks the manifest on the image, and for each artifact in the image, checks for vulns and determines whether the vuln should keep the image from being deployed.
Hope that helps. There are many other workflows but that one is the most direct.
Cheers.
OUTSTANDING comment; excellent questions. Bookmarked. Thanks for this concise high-level infosec punchlist.
This Sir is senior.
Mind Emulation Foundation
"While we are far from understanding how the mind works, most philosophers and scientists agree that your mind is an emergent property of your body. In particular, your body’s connectome. Your connectome is the comprehensive network of neural connections in your brain and nervous system. Today your connectome is biological. "
This is a pretty speculative thesis. It's not at all clear that everything relevant to the mind is found in the connections rather than the particular biochemical processes of the brain. It's a very reductionist view that drastically underestimates the biological complexity of even individual cells. There's a good book, Wetware: A Computer in Every Living Cell, by Dennis Bray going into detail on how much functionality and physical processes are at work even in the most simplest cells that is routinely ignored by these analogies of the brain to a digital computer.
There is this extreme, and I would argue unscientific bias towards treating the mind as something that's recreatable in a digital system probably because it enables this science-fiction speculation and dreams of immortality of people living in the cloud.
Indeed. We humans largely create devices that function either through calculation or through physical reaction, relying on the underlying rules of the universe to "do the math" of, say, launching a cannonball and having it follow a consistent arc. The brain combines both at almost every level. It may be fundamentally impossible to emulate a human personality equal to a real one without a physics simulation of a human brain and its chemistry.
A dragonfly brain takes the input from thirty thousand visual receptor cells and uses it to track prey movement using only sixteen neurons. Could we do the same using an equal volume of transistors?
No one is saying a neuron is a one to one equivalent with a transistor. That behavior does seem like it's possible to emulate with many transistors, however.
Was just talking about quantum cognition and memristors (in context to GIT) a few days ago: https://news.ycombinator.com/item?id=24317768
Quantum cognition: https://en.wikipedia.org/wiki/Quantum_cognition
Memristor: https://en.wikipedia.org/wiki/Memristor
It may yet be possible to sufficiently functionally emulate the mind with (orders of magnitude more) transistors. Though, is it necessary to emulate e.g. autonomic functions? Do we consider the immune system to be part of the mind (and gut)?
Perhaps there's something like an amplituhedron - or some happenstance correspondence - that will enable more efficient simulation of quantum systems on classical silicon pending orders of magnitude increases in coherence and also error rate in whichever computation medium.
For abstract formalisms (which do incorporate transistors as a computation medium sufficient for certain tasks), is there a more comprehensive set than Constructor Theory?
Constructor theory: https://en.wikipedia.org/wiki/Constructor_theory
Amplituhedron: https://en.wikipedia.org/wiki/Amplituhedron
What is the universe using our brains to compute? Is abstract reasoning even necessary for this job?
Something worth emulating: Critical reasoning. https://en.wikipedia.org/wiki/Critical_reasoning
13 Beautiful Tools to Enhance Online Teaching and Learning Skills
"Options for giving math talks and lectures online" https://news.ycombinator.com/item?id=22541754 also lists a number of resources for teaching math online.
How close are computers to automating mathematical reasoning?
It's irresponsible to not mention that a series of results from the 1930s through the 1950s proved that generalized automated proof search is impossible, in the sense that we cannot just ask a computer to find a proof or refutation of Goldbach's Conjecture or other "Goldbach-type" statement and have it do any better than a person. In that sense, no, computers will never fully automate mathematical reasoning, because mathematical reasoning cannot be fully automated.
What we can automate, and have successfully automated many times over, is proof checking. In that sense, yes, computers have already fully automated mathematical reasoning, because checking proofs is fully formalized and mechanized.
I think that what people really ought to be asking for is the ease with which we find new results and communicate them to others. In that sense, computer-aided proofs can be hard to read and so there is much work to be done in making them easier to communicate to humans. Similarly, there is interesting work being done on how to make computer-generated proofs which use extremely-high-level axiom schemata to generate human-like handwaving abstractions.
I'm still not sure how I feel about the ATP/ITP terminology used here. ATPs are either of the weak sort that crunch through SAT, graphs, and other complete-but-hard problems, or the strong sort which are impossible. Meanwhile, folks have drifted from "interactive" to "assistant", and talk of "proof assistants" as tools which, like a human, can write down and look at sections of a proof in isolation, but cannot summon complete arbitrary proofs from its own mind.
Edit: One edit will be quicker than two replies and I tend to be "posting too fast". The main point is captured well by [0], but they immediately link to [1], the central result, which links to [2], an important corollary. At this point, I'm just going to quote WP:
> The first [Gödel] incompleteness theorem states that no consistent system of axioms whose theorems can be listed by an effective procedure (i.e., an algorithm) is capable of proving all truths about the arithmetic of natural numbers. For any such consistent formal system, there will always be statements about natural numbers that are true, but that are unprovable within the system.
> The second [Gödel] incompleteness theorem, an extension of the first, shows that the system cannot demonstrate its own consistency.
> Informally, [Tarski's Undefinability] theorem states that arithmetical truth cannot be defined in arithmetic.
I recognize that these statements may seem surprising, but they are provable and you should convince yourself of them. I recently reviewed [3] and found it to be a very precise and complete introduction to all of the relevant ideas; there's also GEB if you want something more fun.
[0] https://en.wikipedia.org/wiki/Automated_theorem_proving#Deci...
[1] https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...
[2] https://en.wikipedia.org/wiki/Tarski%27s_undefinability_theo...
[3] https://www.logicmatters.net/resources/pdfs/godelbook/GodelB...
As someone unfamiliar with the results you describe, why can’t a machine do better than a human? A smart human is better at writing proofs than a dumb one. If we had a highly advanced general artificial intelligence, why couldn’t it generate better results than the smart humans?
Or is automated proof search impossible for humans as well?
Arguably, humans require more energy per operation. So, presumably such an argument hinges upon what types of operations are performed in conducting automated proof search?
What would "automated proof search" even mean in the context of being performed by a human being?
The context here is that certain problems cannot be solved by an algorithm, which doesn't necessarily translate into "cannot be performed by a machine".
It only means that there cannot be any Turing machine capable of solving them, no matter what "operations" it represents.
The task (in terms of constructor theory) is: Find the functions that sufficiently approximate the observations and record their reproducible derivations.
Either the (unreferenced) study was actually arguing that "automated proof search" can't be done at all, or that human neural computation is categorically non-algorothmic.
Grid search of all combinations of bits that correspond to [symbolic] classical or quantum models.
Or better: evolutionary algorithms and/or neural nets.
Roger Penrose argues that human neural computation is indeed non-algorithmic in nature (see his book "The Emperor's New Mind"; 1989) and speculates that quantum processes are involved.
I don't quite understand the role of neural nets in that context, though. Those just classical computations in the end and should be bound by the same limits that every other algorithms are, shouldn't they?
That human cognition is quantum in nature - that e.g. entanglement is necessary - may be unfalsifiable.
Neuromorphic engineering has expanded since the 1980s. https://en.wikipedia.org/wiki/Neuromorphic_engineering
Quantum computing is the best known method for simulating chemical reactions and thereby possibly also neurochemical reactions. But, Is quantum computing necessary to functionally emulate human cognition?
It may be that a different computation medium can accomplish the same tasks without emulating all of the complexity of the brain.
If the brain is only classical and some people are using their brains to perform quantum computations, there may be something there.
Quantum cognition: https://en.wikipedia.org/wiki/Quantum_cognition
Quantum memristors are still elusive.
From "Quantum Memristors in Frequency-Entangled Optical Fields" (2020) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7079656/ :
> Apart from the advantages of using these devices for computation [12] (such as energy efficiency [13], compared to transistor-based computers), memristors can be also used in machine learning schemes [14,15]. The relevance of the memristor lies in its ubiquitous presence in models which describe natural processes, especially those involving biological systems. For example, memristors inherently describe voltage-dependent ion-channel conductances in the axon membrane in neurons, present in the Hodgkin–Huxley model [16,17].
> Due to the inherent linearity of quantum mechanics, it is not straightforward to describe a dissipative non-linear memory element, such as the memristor, in the quantum realm, since nonlinearities usually lead to the violation of fundamental quantum principles, such as no-cloning theorem. Nonetheless, the challenge was already constructively addressed in Ref. [18]. This consists of a harmonic oscillator coupled to a dissipative environment, where the coupling is changed based on the results of a weak measurement scheme with classical feedback. As a result of the development of quantum platforms in recent years, and their improvement in controllability and scalability, different constructions of a quantum memristor in such platforms have been presented. There is a proposal for implementing it in superconducting circuits [7], exploiting memory effects that naturally arise in Josephson junctions. The second proposal is based on integrated photonics [19]: a Mach–Zehnder interferometer can behave as a beam splitter with a tunable reflectivity by introducing a phase in one of the beams, which can be manipulated to study the system as a quantum memristor subject to different quantum state inputs.
Quantum harmonic oscillators have also found application in modeling financial markets. Quantum harmonic oscillator: https://en.wikipedia.org/wiki/Quantum_harmonic_oscillator
New framework for natural capital approach to transform policy decisions
Natural capital: https://en.wikipedia.org/wiki/Natural_capital
> Natural capital is the world's stock of natural resources, which includes geology, soils, air, water and all living organisms. Some natural capital assets provide people with free goods and services, often called ecosystem services. Two of these (clean water and fertile soil) underpin our economy and society, and thus make human life possible.
Natural capital accounting: https://en.wikipedia.org/wiki/Natural_capital_accounting
> Natural capital accounting is the process of calculating the total stocks and flows of natural resources and services in a given ecosystem or region.[1] Accounting for such goods may occur in physical or monetary terms. This process can subsequently inform government, corporate and consumer decision making as each relates to the use or consumption of natural resources and land, and sustainable behaviour.
Opportunity cost: https://en.wikipedia.org/wiki/Opportunity_cost
> When an option is chosen from alternatives, the opportunity cost is the "cost" incurred by not enjoying the benefit associated with the best alternative choice.[1] The New Oxford American Dictionary defines it as "the loss of potential gain from other alternatives when one alternative is chosen."[2] In simple terms, opportunity cost is the benefit not received as a result of not selecting the next best option. Opportunity cost is a key concept in economics, and has been described as expressing "the basic relationship between scarcity and choice". [3] The notion of opportunity cost plays a crucial part in attempts to ensure that scarce resources are used efficiently.[4] Opportunity costs are not restricted to monetary or financial costs: the real cost of output forgone, lost time, pleasure or any other benefit that provides utility should also be considered an opportunity cost. The opportunity cost of a product or service is the revenue that could be earned by its alternative use.
How do we value essential dependencies in terms of future opportunity costs?
In terms of just mental health?
"National parks a boost to mental health worth trillions: study" https://phys.org/news/2019-11-national-boost-mental-health-w...
> Visits to national parks around the world may result in improved mental health valued at about $US6 trillion (5.4 trillion euros), according to a team of ecologists, psychologists and economists
> Professor Bateman's decision-making framework focuses on the links between the environment and economy and has three components: efficiency, assessing which option generates the greatest benefit; sustainability, the effects of each option on natural capital stocks; and equity, regarding who receives the benefits of a decision and when.
Ian J. Bateman et al. "The natural capital framework for sustainably efficient and equitable decision making", Nature Sustainability (2020). DOI: 10.1038/s41893-020-0552-3 https://www.nature.com/articles/s41893-020-0552-3
Challenge to scientists: does your ten-year-old code still run?
"Ten Simple Rules for Reproducible Computational Research" http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fj... :
> Rule 1: For Every Result, Keep Track of How It Was Produced
> Rule 2: Avoid Manual Data Manipulation Steps
> Rule 3: Archive the Exact Versions of All External Programs Used
> Rule 4: Version Control All Custom Scripts
> Rule 5: Record All Intermediate Results, When Possible in Standardized Formats
> Rule 6: For Analyses That Include Randomness, Note Underlying Random Seeds
> Rule 7: Always Store Raw Data behind Plots
> Rule 8: Generate Hierarchical Analysis Output, Allowing Layers of Increasing Detail to Be Inspected
> Rule 9: Connect Textual Statements to Underlying Results
> Rule 10: Provide Public Access to Scripts, Runs, and Results
... You can get a free DOI for and archive a tag of a Git repo with FigShare or Zenodo.
... re: [Conda and] Docker container images https://news.ycombinator.com/item?id=24226604 :
> - repo2docker (and thus BinderHub) can build an up-to-date container from requirements.txt, environment.yml, install.R, postBuild and any of the other dependency specification formats supported by REES: Reproducible Execution Environment Standard; which may be helpful as Docker Hub images will soon be deleted if they're not retrieved at least once every 6 months (possibly with a GitHub Actions cron task)
BinderHub builds a container with the specified versions of software and installs a current version of Jupyter Notebook with repo2docker, and then launches an instance of that container in a cloud.
“Ten Simple Rules for Creating a Good Data Management Plan” http://journals.plos.org/ploscompbiol/article?id=10.1371/jou... :
> Rule 6: Present a Sound Data Storage and Preservation Strategy
> Rule 8: Describe How the Data Will Be Disseminated
... DVC: https://github.com/iterative/dvc
> Data Version Control or DVC is an open-source tool for data science and machine learning projects. Key features:
> - Simple command line Git-like experience. Does not require installing and maintaining any databases. Does not depend on any proprietary online services. Management and versioning of datasets and machine learning models. Data is saved in S3, Google cloud, Azure, Alibaba cloud, SSH server, HDFS, or even local HDD RAID.
> - Makes projects reproducible and shareable; helping to answer questions about how a model was built.
There are a number of great solutions for storing and sharing datasets.
... "#LinkedReproducibility"
Open textual formats for data and open source application and system software (more precisely, FLOSS), are just as important.
Imagine that x86 - and with it, the PC platform - gets replaced by ARM within a decade. For binary software, this would be a kind of geological extinction event.
The likelihood of there being a [security] bug discovered in a given software project over any significant period of time is near 100%.
It's definitely a good idea to archive source and binaries and later confirm that the output hasn't changed with and without upgrading the kernel, build userspace, execution userspace, and PUT/SUT Package/Software Under Test.
- Specify which versions of which constituent software libraries are utilized. (And hope that a package repository continues to serve those versions of those packages indefinitely). Examples: Software dependency specification formats like requirements.txt, environment.yml, install.R
- Mirror and archive all dependencies and sign the collection. Examples: {z3c.pypimirror, eggbasket, bandersnatch, devpi as a transparent proxy cache}, apt-cacher-ng, pulp, squid as a transparent proxy cache
- Produce a signed archive which includes all requisite software. (And host that download on a server such that data integrity can be verified with cryptographic checksums and/or signatures.) Examples: Docker image, statically-linked binaries, GPG-signed tarball of a virtualenv (which can be made into a proper package with e.g. fpm), ZIP + GPG signature of a directory which includes all dependencies
- Archive (1) the data, (2) the source code of all libraries, and (3) the compiled binary packages, and (4) the compiler and build userspace, and (5) the execution userspace, and (6) the kernel. Examples: Docker can solve for 1-5, but not 6. A VM (virtual machine) can solve for 1-5. OVF (Open Virtualization Format) is an open spec for virtual machine images, which can be built with a tool like Vagrant or Packer (optionally in conjunction with a configuration management tool like Puppet, Salt, Ansible).
When the application requires (7) a multi-node distributed system configuration, something like docker-compose/vagrant/terraform and/or a configuration management tool are pretty much necessary to ensure that it will be possible to reproducibly confirm the experiment output at a different point in spacetime.
A deep dive into the official Docker image for Python
One thing I've learned is that rather than use a docker-entrypoint.sh, most Linux software can be ran just using the `--user 1000:1000` or whatever UID/GID you want to use, as long as you map a volume that can use those permissions. It is a lot cleaner this way.
> Why Tini?
> Using Tini has several benefits:
> - It protects you from software that accidentally creates zombie processes, which can (over time!) starve your entire system for PIDs (and make it unusable).
> - It ensures that the default signal handlers work for the software you run in your Docker image. For example, with Tini, SIGTERM properly terminates your process even if you didn't explicitly install a signal handler for it.
> - It does so completely transparently! Docker images that work without Tini will work with Tini without any changes.
[...]
> NOTE: If you are using Docker 1.13 or greater, Tini is included in Docker itself. This includes all versions of Docker CE. To enable Tini, just pass the `--init` flag to docker run.
https://github.com/krallin/tini#why-tini
I didn't know it was included by default. I'll check it out, thanks!
There are Alpine [1] and Debian [2] miniconda images (within which you can `conda install python==3.8` and 2.7 and 3.4 in different conda envs)
[1] https://github.com/ContinuumIO/docker-images/blob/master/min...
[2] https://github.com/ContinuumIO/docker-images/blob/master/min...
If you build manylinux wheels with auditwheel [3], they should install without needing compilation for {CentOS, Debian, Ubuntu, and Alpine}; though standard Alpine images have MUSL instead of glibc by default, this [4] may work:
echo "manylinux1_compatible = True" > $PYTHON_PATH/_manylinux.py
[4] https://github.com/docker-library/docs/issues/904#issuecomme...
The miniforge docker images aren't yet [5][6] multi-arch, which means it's not as easy to take advantage of all of the ARM64 / aarch64 packages that conda-forge builds now.
[5] https://github.com/conda-forge/docker-images/issues/102#issu...
[6] https://github.com/conda-forge/miniforge/issues/20
There are i686 and x86-64 docker containers for building manylinux wheels that work with many distros: https://github.com/pypa/manylinux/tree/master/docker
A multi-stage Dockerfile build can produce a wheel in the first stage and install that wheel (with `COPY --from=0`) in a later stage; leaving build dependencies out of the production environment for security and performance: https://docs.docker.com/develop/develop-images/multistage-bu...
Interesting! I use miniconda extensively for local development to manage virtual environments for different python versions and love it. I hardly ever actually use the conda packages though.
I assume the main benefit of using these images would be if you are installing from conda repos instead of pip? Otherwise just using the official python images would be as good if not better
Edit: I guess if you needed multiple python versions in a single container this would be a good solution for that as well
Use cases for conda or conda+pip:
- Already-compiled packages (where there may not be binary wheels) instead of requiring reinstallation and subsequent removal of e.g. build-essentials for every install
- Support for R, Julia, NodeJS, Qt, ROS, CUDA, MKL, etc.
- Here's what the Kaggle docker-python Dockerfile installs with conda and with pip: https://github.com/Kaggle/docker-python/blob/master/Dockerfi...
- Build matrix in one container with conda envs
Disadvantages of the official python images as compared with conda+pip:
- Necessary to (re)install build dependencies and a compiler for every build (if there's not a bdist or a wheel for the given architecture) and then uninstall all unnecessary transitive dependencies. This is where a [multi-stage] build of a manylinux wheel may be the best approach.
- No LSM (AppArmor, SELinux, ) for one or more processes in the container (which may have read access to /etc or environment variables and/or --privileged)
- Necessary to build basically everything on non x86[-64] architectures for every container build
Disadvantages of conda / conda+pip:
- Different package repo infrastructure to mirror
- Users complaining that they don't need conda who then proceed to re-download and re-build wheels locally multiple times a day
Additional attributes for comparison:
- The new pip solver (which is slower than the traditional iterative non-solver), conda, and mamba
- repo2docker (and thus BinderHub) can build an up-to-date container from requirements.txt, environment.yml, install.R, postBuild and any of the other dependency specification formats supported by REES: Reproducible Environment Execution Standard; which may be helpful as Docker Hub images will soon be deleted if they're not retrieved at least once every 6 months (possibly with a GitHub Actions cron task)
Quite a few conda packages have patches added by the conda team to help fix problems in packages relying on native code or binaries. Particularly on Windows. If something is available on the primary conda repos it will almost assuredly work with few of any problems cross-platform, whereas pip is hit or miss.
If you’re always on Linux you may never appreciate it but some pip packages are a nightmare to get working properly on Windows.
If you look through the source of the conda repos, you’ll see all kinds of small patches to fix weird and breaking edge cases, particularly in libs with significant C back ends.
Here's the meta.yml for the conda-forge/python-feedstock: https://github.com/conda-forge/python-feedstock/blob/master/...
It includes patches just like distro packages often do.
The Consortium for Python Data API Standards
This actually strikes me as a huuuge waste of effort. I work with every one of the different technologies they mention every day.
The differences and idiosyncrasies are truly, truly not a big deal and really what you want is to allow different library maintainers to do it all differently and just build your own adapters over top of them.
This allows library developers to worry about narrow use cases and have their own separate processes to introduce breaking changes and features that deal with extreme specifics like GPU or TPU interop, heterogeneous distributed backed arrays, jagged arrays, etc. etc.
Let a thousand flowers blossom and a thousand different Python array and record APIs contend.
End users can write their own adapters to mollify irregularities between them, possibly writing different adapters on a case by case basis.
If any “standard” adapters gain popularity as open source projects, great - but don’t try to bake that in from an RFC point of view into the array & data structure libraries themselves. Let them be free / whatever they want to be. That diversity of API approach is super valuable and easily mollified by your own custom adapter logic.
Personally, I like how the Julia ecosystem coalesced around a Tables.jl package that gives some consistency to tabular data packages that choose to implement it. Row-oriented or column-oriented, doesn't matter! Using Tables.jl they can be interchangeable.
There’s a huge difference between “I personally like it when some libraries choose to adhere to a convention” vs “let’s bake this into every tool with a wide, bureaucratic shared RFC process.”
But I don't think people are forced into following right? There's still choice. But if I was developing something in Python, I would rather join than not join.
I’m saying as a consumer of these libraries, I don’t want joiners, I want lots of different APIs that make different trade-offs for each use case, and I will write (or use) adapters if I need to (that’s my responsibility as a consumer).
Just because I personally either do or do not get some value out of a shared API standard doesn’t mean you do or do not, so personal feelings of what one likes need to be set aside, so the service boundary is at the level of consumers choosing their own adapters and making their own trade-offs.
If lots of libraries sign up, we all lose. Much slower compliance-constrained development cycles, less freedom for libraries to break the shared API contract in ways many users would be super happy to abide for the sake of a faster / better feature. Instead of making these trade-offs library by library, case by case, it’s made for you with standardization compliance as the highest constraint, regardless of the user base for which that does / doesn’t work well.
This strikes me as a bit pessimistic. I think you are mostly against _bad_ standards than standards per se.
If the consortium does its job well it will produce a good standard that all the major libraries like: from a UX as well as a performance viewpoint.
If it produces a bad standard, all that will happen is that nobody will sign up. No libraries are going to be _forced_ to do anything.
That’s fair except I’d say even good standards can cause a big slowdown in feature releases, and not everyone values the adherence to the standard as much as the features. It still seems better to just spin out the standard into a set of optional adapters and let library maintainers and users pick their own trade-offs in terms of when to adopt breaking changes, how / whether to smooth over idiosyncrasies across multiple libraries.
Even a good standard has costs and this particular case does not seem like it has good arguments in favor of a wide standard, but many arguments against it.
No, it's easy for library maintainers to offer a compat API in addition to however else they feel they need to differentiate and optimize the interfaces for array operations. People can contribute such APIs directly to libraries once instead of creating many conditionals in every library-utilizing project or requiring yet another dependency on an adapter / facade package that's not kept in sync with the libraries it abstracts.
If a library chooses to implement a spec compatability API, they do that once (optimally, as compared with somebody's hackish adapter facade which has very little comprehension of each library's internals) and everyone else's code doesn't need to have conditionals.
Each of L libraries implements a compat API: O(L)
Each of U library utilizers implements conditionals for every N places arrays are utilized: O(U x N_)
Each of U library utilizers uses the common denominator compat API: O(U)
L < U < (L + U) < (U x N_)
Tech giants let the Web's metadata schemas and infrastructure languish
It's "langushing" and they should do it for us? It's flourishing and they're doing it for us and they have lots of open issues and I want more for free without any work.
Wow! Nobody else does anything to collaboratively, inclusively develop schema and the problem is that search engines aren't just doing it for us?
1) Search engines do not owe us anything. They are not obligated to dominate us or the schema that we may voluntarily decide to include on our pages.
We've paid them nothing. They have no contract for service or agreement with us which compels them to please us or contribute greater resources to an open standard that hundreds of people are contributing to.
2) You people don't know anything about linked data and structured data.
Here's a list of schema: https://lov.linkeddata.es/dataset/lov/ .
Here's the Linked Open Data Cloud: https://lod-cloud.net/
Does your or this publisher's domain include any linked data?
Does this article include any linked data?
Do data quality issues pervade promising, comparatively-expensive, redundant approaches to natural-language comprehension, reasoning, and summarization?
Here, in contributing this example PR adding RDFa to the codeforantarctica web page, I probably made a mistake. https://github.com/CodeForAntarctica/codeforantarctica.githu... . Can you spot the mistake?
There should have been review.
https://schema.org/ClaimReview, W3C Verifiable Claims / Credentials, ld-signatures, and lds-merkleproof2017.
Which brings us to reification, truth values, property graphs, and the new RDF* and SPARQL* and JSON-LD* (which don't yet have repos with ongoing issues to tend to).
3) Get to work. This article does nothing to teach people how to contribute to slow, collaborative schema standards work.
Here's the link to the GitHub Issues so that you can contribute to schema.org: https://github.com/schemaorg/schemaorg
...
"Standards should be better and they should pay for it"
Who are the major contributors to the (W3C) open standard in question?
Is telling them to put up more money or step down going to result in getting what we want? Why or why not?
Who would merge PRs and close issues?
Have you misunderstood the scope of the project? What do the editors of the schema feel in regards to more specific domain vocabularies? Is it feasible or even advisable to attempt to out-schema domain experts who know how to develop and revise an ontology or even just a vocabulary with Protegé?
To give you a sense of how much work goes into creating a few classes and properties defined with RDFS in RDFa in HTML: here's the https://schema.org/Course , https://schema.org/CourseInstance , and https://schema.org/EducationEvent issue: https://github.com/schemaorg/schemaorg/issues/195
Can you find the link to the Use Cases wiki (which was the real work)? What strategy did you use to find it?
...
"Well, Google just does what's good for Google."
Are you arguing that Google.org should make charitable contributions to this project? Is that an advisable or effective way to influence a W3C open standard (where conflicts of interest by people just donating time are disclosed)?
Anyone can use something like extruct or OSDS to extract RDFa, Microdata, and/or JSON-LD from a page.
Everyone can include structured data and linked data in their pages.
There are surveys quantifying how many people have included which types in their pages. Some of that data is included on schema.org types pages.
...
Some written interview questions:
> Which issues have you contributed to? Which issues have you seen all the way to closed? Have you contributed a pull request to the project? Have you published linked data? What is the URL to the docs which explain how to contribute resources? How would you improve them?
https://twitter.com/westurner/status/1291903926007209984
...
After all that's happened here, I think Dan (who built FOAF, which all profitable companies could use instead of https://schema.org/Person ) deserves a week off to add more linked data to the internet now please.
I think that might be fair, but when she makes org came out it pitched itself as trust us we will take care of things, so yeah they don’t owe us anything but track record matters for trust in future ventures by these search engine orgs
schemaorg/schemaorg/CONTRIBUTING.md https://github.com/schemaorg/schemaorg/blob/main/CONTRIBUTIN... explains how you and your organization can contribute resources to the Schema.org W3C project.
If you or your organization can justify contributing one or more people at full or part time due to ROI or goodwill, by all means start sending Pull Requests and/or commenting on Issues.
"Give us more for free or step down". Wow. What PRs have you contributed to justify such demands?
https://schema.org/docs/documents.html links to the releases.
Time-reversal of an unknown quantum state
T-symmetry https://en.wikipedia.org/wiki/T-symmetry > See also links to "reversible computing" but not the "time reversal" disambiguation page?
The "teeter-totter" thought experiment on that page is interesting to me in that it seems to illustrate how the future has more possibilities than the past. But it also occurs to me that the future possibilities are fractal with the past - ie. the toy could fall onto one of many other lower pedestals, teetering as it was before. In this way the "many arbitrary" possibilities could be perceived as anything but.
Electric cooker an easy, efficient way to sanitize N95 masks, study finds
An autoclave is what the electrical cooker is being used to emulate. A pressure cooker is a better option if you'll be on the road, camping, or otherwise without electricity. Researchers in Canada already proved several months ago that N95 masks could be sanitized this way:
Unfortunately the referenced NewsArticle does not link to the ScholarlyArticle https://schema.org/ScholarlyArticle :
"N95 Mask Decontamination using Standard Hospital Sterilization Technologies" (2020-04) https://www.medrxiv.org/content/10.1101/2020.04.05.20049346v... :
> We sought to test the ability of 4 different decontamination methods including autoclave treatment, ethylene oxide gassing, ionized hydrogen peroxide fogging and vaporized hydrogen peroxide exposure to decontaminate 4 different N95 masks of experimental contamination with SARS-CoV-2 or vesicular stomatitis virus as a surrogate. In addition, we sought to determine whether masks would tolerate repeated cycles of decontamination while maintaining structural and functional integrity. We found that one cycle of treatment with all modalities was effective in decontamination and was associated with no structural or functional deterioration. Vaporized hydrogen peroxide treatment was tolerated to at least 5 cycles by masks. Most notably, standard autoclave treatment was associated with no loss of structural or functional integrity to a minimum of 10 cycles for the 3 pleated mask models. The molded N95 mask however tolerated only 1 cycle. This last finding may be of particular use to institutions globally due to the virtually universal accessibility of autoclaves in health care settings.
The ScholarlyArticle referenced by and linked to by the OP NewsArticle is "Dry Heat as a Decontamination Method for N95 Respirator Reuse" (2020-07) https://pubs.acs.org/doi/full/10.1021/acs.estlett.0c00534 . Said article does not reference "N95 Mask Decontamination using Standard Hospital Sterilization Technologies" DOI: 10.1101/2020.04.05.20049346v2 . We would do well to record that (article A, seemsToConfirm, Article B) as third-party linked data (only if both articles do specifically test the efficacy of the given sterilization method with the COVID-19 coronavirus)
None of this is necessary for non-healthcare workers, since the virus will die off on a fabric mask in a few hours.
1) Not a single study has demonstrated that viable Sars-Cov-2 virus survives on porous materials in the real world
2) Even before Covid, it was known for decades (common medical knowledge) that human coronaviruses and flu viruses do not remain viable on porous materials for more than several hours.
That this common knowledge is not so common knowledge among the public is a failure of public health communication. The one or two alarmist studies showing that the virus "survives" X number of days don't reflect the real world because 1) the researchers literally directly douse or soak the surface with a huge viral load, and 2) the researchers usually only look for viral genetic material, not whether the virus can infect cells. Viability != detecting viral genetic material (same story for people - we shed viral genetic material long after we stop being contagious).
There is a reason the CDC has always said surface transmission is rare: https://www.nytimes.com/2020/05/22/health/cdc-coronavirus-to... there have been no documented cases of surface transmission: https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-si... Fortunately some in the media are catching on to the hygiene (security) theater: https://www.theatlantic.com/ideas/archive/2020/07/scourge-hy...
Citations for 1) and 2):
Most studies use unrealistic starting doses: https://www.thelancet.com/pdfs/journals/laninf/PIIS1473-3099...
Other human coronaviruses don't survive long on porous surfaces (gone by 6 hours): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7134510/. A hospital randomly collected patient and real surface swabs (including non-porous surfaces) for original SARS, a minority were PCR positive, none of the swabs were infectious (viral culture): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7134510/. Same story for flu (virus can be detected for days, but inactive and not viable after a few hours): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3222642/.
I could only find this study on Sars-Cov-2 that cultured the virus (still used a huge viral dose in a lab setting, not the real world). Even though they were able to culture the virus, only 1% of virus remains after 6 hours on a surgical mask, several orders of magnitude less for cotton clothing: https://www.medrxiv.org/content/10.1101/2020.05.07.20094805v...
This long post was originally meant to be a reply to someone asking for a citation; it's yet another example of how much effort is required to combat misinformation.
"Interim Recommendations for U.S. Households with Suspected or Confirmed Coronavirus Disease 2019 (COVID-19)" https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-si... :
> On the other hand, transmission of novel coronavirus to persons from surfaces contaminated with the virus has not been documented. Recent studies indicate that people who are infected but do not have symptoms likely also play a role in the spread of COVID-19. Transmission of coronavirus occurs much more commonly through respiratory droplets than through objects and surfaces, like doorknobs, countertops, keyboards, toys, etc. Current evidence suggests that SARS-CoV-2 may remain viable for hours to days on surfaces made from a variety of materials. Cleaning of visibly dirty surfaces followed by disinfection is a best practice measure for prevention of COVID-19 and other viral respiratory illnesses in households and community settings
Fed announces details of new interbank service to support instant payments
If anyone from the FED is reading this I have this advice: don't use ISO20022! (I've been working on the brazilian instant payments project, known as PIX). It's an overly-complicated-try-to-solve-everything standard that promises interoperability but only delivers pain and misery. Since each country has its own standards for person and account identification, in dealing with these local realities, the ISO20022 adds quite a lot of complexity. But all this cost does not result in interoperability. In fact, each payment scheme end up with a localized version of the standard, incompatible with others. What could have been a clean API, ends up a mess.
What alternative do you suggest? Keeping in mind all the parties that will be involved, besides your project.
Create a fit to purpose, simpler, protocol using a good interface specification language (OpenAPI or protocol buffers). Maybe borrow concepts and patterns from the ISO standard, but not adopt it. It will not deliver its promise of interoperability. It might actually make it harder to interoperate with other payment schemes.
Interledger Protocol (ILP, ILPv4).
Interledger Architecture:
https://interledger.org/rfcs/0001-interledger-architecture/#... :
> For purposes of Interledger, we call all settlement systems ledgers. These can include banks, blockchains, peer-to-peer payment schemes, automated clearing house (ACH), mobile money institutions, central-bank operated real-time gross settlement (RTGS) systems, and even more.
[...]
> Interledger provides for secure payments across multiple assets on different ledgers. The architecture consists of a conceptual model for interledger payments, a mechanism for securing payments, and a suite of protocols that implement this design.
> The Interledger Protocol (ILP) is the core of the Interledger protocol suite. Colloquially, the whole Interledger stack is sometimes referred to as "ILP". Technically, however, the Interledger Protocol is only one layer in the stack.
> Interledger is not a blockchain, a token, nor a central service. Interledger is a standard way of bridging financial systems. The Interledger architecture is heavily inspired by the Internet architecture described in RFC 1122, RFC 1123 and RFC 1009.
[...]
> You can envision the Interledger as a graph where the points are individual nodes and the edges are accounts between two parties. Parties with only one account can send or receive through the party on the other side of that account. Parties with two or more accounts are connectors, who can facilitate payments to or from anyone they're connected to.
> Connectors [AKA routers] provide a service of forwarding packets and relaying money, and they take on some risk when they do so. In exchange, connectors can charge fees and derive a profit from these services. In the open network of the Interledger, connectors are expected to compete among one another to offer the best balance of speed, reliability, coverage, and cost.
ILP > Peering, Clearing and Settling: https://interledger.org/rfcs/0032-peering-clearing-settlemen...
ILP > Simple Payment Setup Protocol (SPSP): https://interledger.org/rfcs/0009-simple-payment-setup-proto...
> This document describes the Simple Payment Setup Protocol (SPSP), a basic protocol for exchanging payment information between payee and payer to facilitate payment over Interledger. SPSP uses the STREAM transport protocol for condition generation and data encoding.
> (Introduction > Motivation) STREAM does not specify how payment details, such as the ILP address or shared secret, should be exchanged between the counterparties. SPSP is a minimal protocol that uses HTTPS for communicating these details.
[...]
GET /.well-known/pay HTTP/1.1
Host: example.com
Accept: application/spsp4+json, application/spsp+jsonShrinking deep learning’s carbon footprint
"Unlearning" is one algorithmic approach that may yield substantial energy consumption gains.
With many deep learning models, it's not possible to determine when or from what source something was learned: it's not possible to "back out" a change to the network and so the whole model has to be re-trained from scratch; which is O(n) instead of O(1.x).
The article covers software approaches (more energy-efficient algorithms) and mentions GPUs but not TPUs or ASICs.
Specialized chips (built with dynamic fabrication capacities) are far more energy efficient for specific types of workloads. We see this with mining ASICs, SSL accelerators, and also with Tensor Processing Units (for deep learning).
The externalities of energy production are the ultimate concern. If you're using cheap, clean energy with minimized external costs ("sustainable energy"), the energy-efficiency of the algorithm and the chips is of much less concern.
Could we recognize products, services, and data centers that were produced with and/or run on directly sourced clean energy as "200% Green"; with a logo on the box and/or the footer? 100% offset by PPAs is certainly progress.
Show HN: Starboard – Fully in-browser literate notebooks like Jupyter Notebook
Hi HN, I developed Starboard over the past months.
Cell-by-cell notebooks like Jupyter are great for prototyping, explaining and exploration, but their dependence on a Python server (with often undocumented dependencies) limits their ability to be shared and remixed. Now that browsers support dynamic imports, it has become possible to create a similar workflow entirely in the browser.
That motivated me to build Starboard Notebook, a tool I wished existed. It's:
* Run entirely in the browser, there is no server or setup, it's all static files.
* Web-native, so no widget system is necessary. There is nearly no magic, it's all web tech (HTML, CSS, JS).
* Stores as a plaintext file, it will play nicely with version control systems.
* Hackable: the sandbox that your code gets run in contains the editor itself, so you can metaprogram the editor itself (e.g. adding support for other languages such as Python through WASM).
* Open source (https://github.com/gzuidhof/starboard-notebook).
You can import any code that targets the browser directly (e.g. puts stuff on the window object), or that has exports in ES module format.
I'm happy to answer any questions!
Neat! There's a project called Jyve that compiles Jupyter Lab to WASM (using iodide). https://github.com/deathbeds/jyve There are kernels for JS, CoffeeScript, Brython, TypeScript, and P5. FWIU, the kernels are marked as unsafe because, unfortunately, there seems to be no good way to sandbox user-supplied notebook code from the application instance. The README describes some of the vulnerabilities that this entails.
The jyve project issues discuss various ideas for repacking Python packages beyond the set already included with Pyodide and supporting loading modules from remote sources.
https://developer.mozilla.org/en-US/docs/Web/Security/Subres... : "Subresource Integrity (SRI) is a security feature that enables browsers to verify that resources they fetch (for example, from a CDN) are delivered without unexpected manipulation. It works by allowing you to provide a cryptographic hash that a fetched resource must match."
There's a new Native Filesystem API: "The new Native File System API allows web apps to read or save changes directly to files and folders on the user's device." https://web.dev/native-file-system/
We'll need a way to grant specific URLs specific, limited amounts of storage.
https://github.com/iodide-project/pyodide :
> The Python scientific stack, compiled to WebAssembly
> [...] Pyodide brings the Python 3.8 runtime to the browser via WebAssembly, along with the Python scientific stack including NumPy, Pandas, Matplotlib, parts of SciPy, and NetworkX. The packages directory lists over 35 packages which are currently available.
> Pyodide provides transparent conversion of objects between Javascript and Python. When used inside a browser, Python has full access to the Web APIs.
https://github.com/deathbeds/jyve/issues/46 :
> Would miniforge and conda-forge build a WASM architecture target?
> Emscripten or WASI?
Ask HN: Learning about distributed systems?
I used to love Operating Systems during my undergrads, Modern Operating Systems by Tanenbaum is till date the only academic book I've read entirely. I recently read an article about how Amazon built Aurora by Werner Vogels and I was captivated by it. I want to start reading about Distributed Systems. What would be a good start/Road Map?
The book “Designing Data-Intensive Applications” by Martin Kleppman is a fantastic read with such a concise train of thought. It builds up from basics, adds another thing, and another thing.
I kept asking myself, what would happen if I were to extend on the feature currently presented in the chapter I was reading, only to find out my answers in the next chapter.
Brilliant book
> "Designing Data-Intensive Applications” by Martin Kleppman: https://dataintensive.net/ https://g.co/kgs/xJ73FS
From a previous question re: "Ask HN: CS papers for software architecture and design?" (https://news.ycombinator.com/item?id=15778396 and distributed systems we eventually realize were needed in the first place:
> Bulk Synchronous Parallel: https://en.wikipedia.org/wiki/Bulk_synchronous_parallel .
Many/most (?) distributed systems can be described in terms of BSP primitives.
> Paxos: https://en.wikipedia.org/wiki/Paxos_(computer_science) .
> Raft: https://en.wikipedia.org/wiki/Raft_(computer_science) #Safety
> CAP theorem: https://en.wikipedia.org/wiki/CAP_theorem .
Papers-we-love > Distributed Systems: https://github.com/papers-we-love/papers-we-love/tree/master...
awesome-distributed-systems also has many links to theory: https://github.com/theanalyst/awesome-distributed-systems
- Byzantine fault: https://en.wikipedia.org/wiki/Byzantine_fault :
> A [Byzantine fault] is a condition of a computer system, particularly distributed computing systems, where components may fail and there is imperfect information on whether a component has failed. The term takes its name from an allegory, the "Byzantine Generals Problem",[2] developed to describe a situation in which, in order to avoid catastrophic failure of the system, the system's actors must agree on a concerted strategy, but some of these actors are unreliable.
awesome-bigdata lists a number of tools: https://github.com/onurakpolat/awesome-bigdata
Practically, dask.distributed (joblib -> SLURM,), dask ML, dask-labextension (a JupyterLab extension for dask), and the Rapids.ai tools (e.g. cuDF) scale from one to many nodes.
Not without a sense of irony, as the lists above list many papers that could be readings with quizzes,
Distributed systems -> Distributed computing: https://en.wikipedia.org/wiki/Distributed_computing
Category: Distributed computing: https://en.wikipedia.org/wiki/Category:Distributed_computing
Category:Distributed_computing_architecture : https://en.wikipedia.org/wiki/Category:Distributed_computing...
DLT: Distributed Ledger Technology: https://en.wikipedia.org/wiki/Distributed_ledger
Consensus (computer science) https://en.wikipedia.org/wiki/Consensus_(computer_science)
Ask HN: How can I “work-out” critical thinking skills as I age?
As I get older, I realized I’m not as sharp as I used to be. Maybe it’s from the fatigue of juggling 2 kids, but I’m very ill prepared for interviews because I simply can’t answer “product questions” and brain teasers. It’s a skill I need, and truthfully I was never good at consultant type questions to begin with but I’m seeing a lot of these questions in Data Science interviews.
Any help or resources will be tremendously appreciated.
Problem solving: https://en.wikipedia.org/wiki/Problem_solving
Critical thinking: https://en.wikipedia.org/wiki/Critical_thinking
Computational Thinking: https://en.wikipedia.org/wiki/Computational_thinking
> 1. Problem formulation (abstraction);
> 2. Solution expression (automation);
> 3. Solution execution and evaluation (analyses).
Interviewers may be more interested in demonstrating problem solving methods and f thinking aloud than an actual solution in an anxiety-producing scenario.
https://en.wikipedia.org/wiki/Brilliant_(website) ;
> Brilliant offers guided problem-solving based courses in math, science, and engineering, based on National Science Foundation research supporting active learning.[14]
Coding Interview University: https://github.com/jwasham/coding-interview-university
Programmer Competency Matrix: https://github.com/hltbra/programmer-competency-checklist
Inference > See also: https://en.wikipedia.org/wiki/Inference
- Deductive reasoning: https://en.wikipedia.org/wiki/Deductive_reasoning
- Inductive reasoning: https://en.wikipedia.org/wiki/Inductive_reasoning
> This is the [open] textbook for the Foundations of Data Science class at UC Berkeley: "Computational and Inferential Thinking: The Foundations of Data Science" http://inferentialthinking.com/
The tragedy of FireWire: Collaborative tech torpedoed by corporations
Due to DMA (Direct Memory Access) in most implementations, IEEE 1394 ("FireWire") can be used to directly read from and write to RAM.
See: IEEE 1394 > Security issues https://en.wikipedia.org/wiki/IEEE_1394#Security_issues
FWIU, USB 3 is faster than FireWire; there are standard, interchangeable USB connectors and adapters; and USB implementations do not use DMA. https://en.wikipedia.org/wiki/USB_3.0
> FWIU, USB 3 is faster than FireWire
You've missed the point. The title of the article reads "The tragedy of FireWire" - how a good standard was victimized due to corporate politics. FireWire was 10 years ahead of USB. In 2002, FireWire 800 provided a data rate of 800 Mbps, full-duplex. On the other hand, USB 2 only had 480 Mbps, half-duplex. USB didn't catch up until 2010, after USB 3.0 was released. FireWire uses 30 V, 1.5 A power, and it supports power delivery up to 45 watts. USB didn't offer anything similar, not even the latest USB 3, until 2014 after Power Delivery and Type-C's standardization - and real applications only started to ramp up by ~2017. Both limitations of USB 2 were detrimental to some applications, e.g. using hard drives on a PC was painful due to speed and power constraints, had FireWire became more common, it would not be an issue. FireWire can also do everything Ethernet can do (in fact, 1394b's signaling is better than 100Base-TX Ethernet, and has lower radiation), networking is natively supported, you can network two computers together directly with a 1394 cable! On USB, it's only possible using an external controller. One application was NAS, a hard drive can be shared directly over 1394, and mounted as a local drive. Not possible with USB. Finally, 1394b does not have USB's 5-meter limitation due to protocol timing, and natively supports signaling over CAT-5 and fiber optics cable for a distance of ~50-100 meters - on USB, it requires a media converter running vendor-specific custom protocols and not interoperable. All of these features are standardized in 2004 in IEEE 1394b.
Of course, limitations are not inherently USB 2's failures, they have different applications, FireWire was more expensive, and USB's goal was low-cost. But the tragedy was that bad business decision prevented FireWire to achieve its full potential, slowly fading out, and eventually became irrelevant after USB 3 despite its initial good engineering.
FireWire was a tragedy.
> Due to DMA (Direct Memory Access) in most implementations, IEEE 1394 ("FireWire") can be used to directly read from and write to RAM.
This is why we need IOMMU.
You should not blame IEEE 1394 for DMA attacks just because it supports DMA. I agree that the tragedy of FireWire probably prevented a common PC exploit vector as an lucky unintentional consequence, but it's just a symptom, not the actual issue - and the symptom has came back in another form.
Most sufficiently fast hardware interfaces support DMA, and they're equally vulnerable - this includes PCI-E, ExpressCard (which exposes PCI-E), USB 4 (which exposes PCI-E port), Thunderbolt (which exposes PCI-E). Practical exploits are already widespread, not limited to arbitrary memory reads/writes, but also OptionROM arbitrary code execution if a Thunderbolt device is plugged in at boot time.
And this is not only a problem for external ports like 1394, USB 4 or Thunderbolt. Threats also exist from internal devices, like a Ethernet or Wi-Fi controller on the motherboard. While you cannot initiate a DMA transaction via Ethernet (without RDMA) or Wi-Fi since they don't expose PCI-E, the fact that those are connected directly to PCI-E implies any vulnerability in the controller firmware potentially allows an attacker to launch a DMA attack. Exploits already exist for Wi-Fi controllers.
This is the real problem. While the mechanism of IOMMU exists, CPU manufacturers and operating systems previously did little to systematically protect the system from DMA. In the past, Intel intentionally removed IOMMU functionality from low-end CPUs and motherboard chipsets in order to sell more Xeon CPUs to enterprise users. Also, while IOMMU is supported by operating systems, kernel code and drivers were not systematic audited for security - serious IOMMU bypass vulnerabilities have been previously discovered (and patched) in macOS and Linux's IOMMU implementations.
With continued proliferation of USB 4 and Thunderbolt, the lack of IOMMU and buggy driver implementation will become a bigger problem.
So far, QubesOS is most most protected operating system from DMA attacks. Untrusted hardware are assigned to a dedicated VM, and IOMMU is enforced by the hypervisor, not the individual device drivers in operating systems. Compromising a device driver in an untrusted domain does not directly expose the rest of QubesOS to attackers.
> and USB implementations do not use DMA.
USB 4 does now.
So your argument is that not security but cost is the reason that USB "won" the external device interface competition with FireWire?
Good to know that USB4 implementations are making the same mistake as FireWire implementors did in choosing performance over security . Unfortunately it looks like there will be no alternative except for maybe to use a USB3 hub (or an OS with fuzzed IOMMU and also controller firmwares)?
Could an NX bit for data coming from buses with and without DMA help at all?
Hot gluing external ports now seems a bit more rational and justified for systems where physical access is less controlled.
> So your argument is that not security, but cost is the reason that USB "won" the external device interface competition with FireWire?
Sorry, but I heavily suspect that you didn't read the original article and I suggest you to do so.
Security has nothing to do with 1394's market adoption, in the 2010s, some desktops still have 1394, and some laptops still have ExpressCard, and you don't see anyone complains about security, only some security researchers.
The reason was partially cost, but also a bad business decision by Steve Jobs, which made Intel and Microsoft stopped their efforts for pushing 1394, largely damaged it before it even got a chance for wider adoption.
> FireWire's principal creator, Apple, nearly killed it before it could appear in a single device. And eventually the Cupertino company effectively did kill FireWire, just as it seemed poised to dominate the industry.
> Intel sent its CTO to talk to Jobs about the change, but the meeting went badly. Intel decided to withdraw its support for FireWire—to pull the plug on efforts to build FireWire into its chipsets—and instead throw its weight behind USB 2.0.
> Sirkin believes that Microsoft could have reversed the new licensing policy by citing the prior signed agreement. "Microsoft must have thrown it away," he speculated, because it would have "stopped Apple in its tracks."*
--
> Good to know that USB4 implementations are making the same mistake as FireWire implementors did in choosing performance over security.
DMA is optional and can be disabled.
Speaking of usability, DMA is limited to the Thunderbolt part (which is ultimately the PCI-E part) of USB 4 standard. You don't have to use USB 4's PCI-E. Unless the USB device really needs PCI-E, like RAID storage box or a GPU, disabling DMA is not a problem. Requiring explicit user consent for PCI-E is also a solution, operating systems already does it for Thunderbolt devices. Same treatment can be applied to USB 4.
And again, calling out DMA support is only shooting the messenger while ignoring the underlying problem. DMA is essential for any high-performance system bus, and it can a problem regardless of who supported it, or whether it's connected to an external port. A Wi-Fi controller handles untrusted data, but despite being an internal device on the motherboard, it's still potentially an exploit vector for DMA attackers.
> (or an OS with fuzzed IOMMU and also controller firmwares)
Only IOMMU is needed (and possibly a good driver, but not strictly required, see QubesOS). If your IOMMU is good, you don't have to trust firmware or hardware, since arbitrary DMAs are blocked by IOMMU, just like how memory protection (MMU) works but for I/O. Memory spaces are isolated.
The IOMMU vulnerabilities I previous mentioned are already patched, and a complete bypass of IOMMU is unlikely to occur in the future. Driver bugs are still an issue, more RAM regions can be exposed by drivers than what's really needed, and potentially you can pretend to be any hardware and feed bad data to trick the most vulnerable driver to expose more RAM regions, and I expect to see more bugs.
Turn on IOMMU, and apply OS patches often, and you'll probably be fine.
I read much of the article (which assumed that "FireWire" failed because of issues with suppliers failing to work together instead of waning demand (due in part to corporate customers' knowledge of the security risks of most implementations)).
Thanks for the info on USB-4, DMA, IOMMU.
IOMMU: https://en.wikipedia.org/wiki/Input%E2%80%93output_memory_ma...
Looks like there are a number of iommu Linux kernel parameters: https://www.kernel.org/doc/html/latest/admin-guide/kernel-pa...
Wonder what the defaults are and what the comparable parameters are for common consumer OSes.
Looks like NX bit support is optional in IOMMUs.
Can I configure the amount of RAM allocated to this?
> Wonder what the defaults
See the Thunderclap FAQ, most of your questions are explained. Thunderclap is the DMA attack on Thunderbolt, and everything should be applicable to USB 4 as well, since USB 4 incorporated Thunderbolt directly.
From the FAQ,
> In macOS 10.12.4 and later, Apple addressed the specific network card vulnerability we used to achieve a root shell. However the general scope of our work still applies; in particular that Thunderbolt devices have access to all network traffic and sometimes keystrokes and framebuffer data.
> Microsoft have enabled support for the IOMMU for Thunderbolt devices in Windows 10 version 1803, which shipped in 2018. Earlier hardware upgraded to 1803 requires a firmware update from the vendor. This brings them into line with the baseline for our work, however the more complex vulnerabilities we describe remain relevant.
> Recently, Intel have contributed patches to version 5.0 of the Linux kernel (shortly to be released) that enable the IOMMU for Thunderbolt and prevent the protection-bypass vulnerability that uses the ATS feature of PCI Express.
But note that Microsoft says on its website,
> This feature does not protect against DMA attacks via 1394/FireWire, PCMCIA, CardBus, ExpressCard, and so on.
Basically, IOMMU is enabled by default in most systems now. However, only on Thunderbolt devices (and USB 4 devices, since Thunderbolt is now its subset). But not on other PCI-E ports, such as the internal PCI-E ports, or external ones like 1394 or ExpressCard. I think it's due to fears of performance and compatibility issues, there's no reason why it cannot be implemented if one insists. Also, from the macOS example, you see that bad drivers can be a problem, even with IOMMU, if a driver is bad, attackers can exploit them and trick them to give more RAM access (If my memory is correct, Linux's implementation was better, but since it's an early FAQ, I don't know if macOS has improved).
> what the comparable parameters are for common consumer OSes.
On the Linux kernel, with an Intel CPU, the kernel option "intel_iommu=on" will enable IOMMU completely for everything, internal and external, which is the most ideal option. You should see
DMAR: IOMMU enabled
IOMMU needs support from the CPU and motherboard chipset, Intel's IOMMU is called VT-D and marketed it as I/O virtualization for virtual machines. Most i3, i5, and i7 CPUs are supported. You may need to turn it on in BIOS if there's an option. It should work on most laptops, even an old Ivy Bridge laptop should work - which is good news, laptops are the most vulnerable platform. Unfortunately, on desktops, Intel intentionally removed IOMMU from high-end CPUs preferred by PC enthusiasts in the 1st (Nehalem), 2nd (Sandy Bridge), 3rd (Ivy Bridge), and 4th (Haswell) generations, possibly as a strategy to sell more Xeon chips to enterprise virtualization users (likely in line with their cripple-ECC strategy). Most high-end overclockable K-series CPUs have their IOMMU removed, while the non-K counterparts are supported. Also, in these generations, some motherboard chipsets are crippled and don't have IOMMU. Fortunately, Intel no longer does it since 5/6th gen (Broadwell/Skylake) CPUs.
No experience on AMD yet, but it should be similar. I think possibly better, my impression is that AMD doesn't intentional crippling IOMMU or ECC like Intel does (a nasty strategy...). Check the docs.
The Developer’s Guide to Audit Logs / SIEM
This article suggests that there should be separate data collection systems for: analytics, SIEM logs, and performance metrics.
The article mentions the CEF (Common Event Format) standard but not syslog or GELF or other JSON formats.
[ArcSight] Common Event Format [PDF]: https://kc.mcafee.com/resources/sites/MCAFEE/content/live/CO...
GELF: Graylog Extended Log Format: https://docs.graylog.org/en/latest/pages/gelf.html
Wikipedia > Syslog lists a few limitations of Syslog (no message delivery confirmation, though there is a reliable delivery RFC; and insufficient payload standardization) and also links to the existing Syslog RFCs. https://en.wikipedia.org/wiki/Syslog
Are push-style systems ideal for security logshipping systems? What sort of a message broker is ideal? AMQP has reliable delivery; while, for example, ZeroMQ does not and will drop messages due to resource exhaustion.
Developers simply need an API for their particular framework to non-blockingly queue and then log structs to a remote server. This typically means moving beyond a single-threaded application architecture so that the singular main [green] thread is not blocked when the remote log server is not responding.
SIEM: Security information and event management: https://en.wikipedia.org/wiki/Security_information_and_event...
Del.icio.us
The Firefox (and Chromium) bookmarks storage and sync systems still don't persist tags!
"Allow reading and writing bookmark tags" https://bugzilla.mozilla.org/show_bug.cgi?id=1225916
Notes re: how this could be standardized with JSON-LD: https://bugzilla.mozilla.org/show_bug.cgi?id=1225916#c116
The existing Web Experiment for persisting bookmark tags: https://github.com/azappella/webextension-experiment-tags/bl...
Ask HN: Recommendations for Books on Writing?
I want to propose a book club for writing as an engineer. Writing is fundamentally and critically important, but it seems that we don't emphasize it as much as we should for engineers (outside Amazon, where apparently it is a prominent member of the leadership pantheon).
I'm interested in any suggestions that HN has for great books on writing as an engineer! Accessibility and ease are important factors for a book club as well.
Technical Writing: https://en.wikipedia.org/wiki/Technical_writing
Google Technical Writing courses (1 & 2) and resources: https://developers.google.com/tech-writing :
- Google developer documentation style guide: https://developers.google.com/style
- Microsoft Writing Style Guide: https://docs.microsoft.com/en-us/style-guide/welcome/
Season of Docs is a program where applicants write documentation for open source projects: https://developers.google.com/season-of-docs/
Many open source projects are happy to accept necessary contributions of docs and editing; but do keep in mind that maintaining narrative documentation can be far more burdensome than maintaining API documentation that's kept next to the actual code. Systems like doxygen, epidoc, javadoc, and sphinx-apidoc enable developers to generate API documentation for a particular version of the software project as one or more HTML pages.
ReadTheDocs builds documentation from ReStructuredText and now also Markdown sources using Sphinx and the ReadTheDocs Docker image. ReadTheDocs organizes docs with URLs of the form <projectname>.rtfd.io/<language>/<version|latest>: https://docs.readthedocs.io/en/latest/ . The ReadTheDocs URL scheme reduces the prevalence of broken external links to documentation; though authors are indeed free to delete and rename docs pages and change which VCS tags are archived with RTD.
Write the Docs is a conference for technical documentation authors which is supported in part by ReadTheDocs: https://www.writethedocs.org/
Write the Docs > Learning Resources > All our videos and articles: https://www.writethedocs.org/topics/ :
> This page links to the topics that have been covered by conference talks or in the newsletter.
You might say that UX (User Experience) includes UI design and marketing: the objective is to imagine yourself as a customer experiencing the product or service afresh.
Writing dialogue is an activity we often associate more with creative writing exercises; where the objective is to meditate upon compassion for others.
One must imagine themself as ones/people/persons who interact with the team.
Cognitive walkthrough: https://en.wikipedia.org/wiki/Cognitive_walkthrough
The William Golding, Jung, and Joseph Campbell books on screenwriting, archetypes, and the hero's journey monomyth are excellent; if you're looking for creative writing resources.
Ask HN: How did you learn x86-64 assembly?
I'm an experienced C/C++ programmer and I occasionally look at the generated assembly to check for optimizations, loop unrolling, vectorization, etc. I understand what's going on the surface level, but I have a hard time understand what's going on in detail, especially with high optimization levels, where the compiler would do all kinds of clever tricks. I experiment with code in godbolt.org and look up the various opcodes, but I would like to take a more structured way of learning x86-64 assembly, especially when it comes to common patterns, tips and tricks, etc.
Are there any good books or tutorials you can recommend which go beyond the very beginner level?
High Level Assembly (HLA) https://en.wikipedia.org/wiki/High_Level_Assembly
> HLA was originally conceived as a tool to teach assembly language programming at the college-university level. The goal is to leverage students' existing programming knowledge when learning assembly language to get them up to speed as fast as possible. Most students taking an assembly language programming course have already been introduced to high-level control flow structures, such as IF, WHILE, FOR, etc. HLA allows students to immediately apply that programming knowledge to assembly language coding early in their course, allowing them to master other prerequisite subjects in assembly before learning how to code low-level forms of these control structures. The book The Art of Assembly Language Programming by Randall Hyde uses HLA for this purpose
Web: https://plantation-productions.com/Webster/
Book: "The Art of Assembly Language Programming" https://plantation-productions.com/Webster/www.artofasm.com/
Portable, Opensource, IA-32, Standard Library: https://sourceforge.net/projects/hla-stdlib/
"12.4 Programming in C/C++ and HLA" in the Linux 32 bit edition: https://plantation-productions.com/Webster/www.artofasm.com/...
... A chapter(s) about wider registers, WASM, and LLVM bitcode etc might be useful?
... Many awesome lists link to OllyDbg and other great resources for ASM; like such as ghidra: https://www.google.com/search?q=ollydbg+site%3Agithub.com+in...
Brain connectivity levels are equal in all mammals, including humans: study
This is outrageously misleading. To make a claim about the number of synapses between neurons based on MRI data is completely unwarranted. Voxel size (single volumetric pixel) in MRI is approximately 1mm, while synapse size is way less than a micron. You need resolution per pixel on the order of 10s of nanometers to identify synapses.
I wouldn’t be surprised if a dead salmon also has “equal” connectivity: https://www.wired.com/2009/09/fmrisalmon/
Surely there's something to learn here though. I haven't read the original paper but a quantity that's preserved across brain scales is either an artifact or a neat insight.
Your criticism reads like someone accusing economists of being outrageously misleading when they don't sample individual households but measure macro indicators. It's like saying Ramon y cajal was ridiculous because he couldn't image the neuropil effectively. Or like saying early optogenetics experiments were ridiculous because who knows if you're stimulating a neuron in a realistic manner?
And in any case, it's true that synapses are comically small relative to voxel size, but we also have some reasonable information about projection patterns and synapse number from various tracer or rabies studies with which you are no doubt familiar.
I haven't read the nature paper the press release is about and I'm not a huge fan of many d/fMRI practices or derived claims. And I've worked with enough mammalian dwi data to be skeptical of specific connection claims. But this strikes me as a rather interesting result even if you can't measure all the synapses at the right resolution: either the tractography method has connectivity conservation artifacts baked in, or there's something interesting going on.
"fNIRS Compared with other neuroimaging techniques" https://en.wikipedia.org/wiki/Functional_near-infrared_spect...
> When comparing and contrasting these devices it is important to look at the temporal resolution, spatial resolution, and the degree of immobility.
I think I missed your overall point?
OP suggests that the spatial resolution of existing MRI neuroimaging capabilities is insufficient to observe or so characterize or so generalize about neuronal activity in mammalian species. fNIRS (functional near-infrared spectroscopy) is one alternative neuroimaging capability that we could compare fMRI with according to the criteria for comparison suggested in the cited Wikipedia article: "temporal resolution, spatial resolution, and the degree of immobility".
Ask HN: Resources to start learning about quantum computing?
Hi there,
I'm an experienced software engineer (+15 years dev experience, MsC in Computer Science) and quantum computing is the first thing in my experience that is being hard to grasp/understand. I'd love to fix that ;)
What resources would you recommend to start learning about quantum computing?
Ideally resources that touch both the theoretical base and evolve to more practical usages.
"What are some good resources to learn about Quantum Computing?" https://news.ycombinator.com/item?id=16052193 https://westurner.github.io/hnlog/#comment-16052193
Launch HN: Charityvest (YC S20) – Employee charitable funds and gift matching
Stephen, Jon, and Ashby here, the co-founders of Charityvest (https://charityvest.org). We created a modern, simple, and affordable way for companies to include charitable giving in their suite of employee benefits.
We give employees their own tax-deductible charitable giving fund, like an “HSA for Charity.” They can make contributions into their fund and, from their fund, support any of the 1.4M charities in the US, all on one tax receipt.
Using the funds, we enable companies to operate gift matching programs that run on autopilot. Each donation to a charity from an employee is matched automatically by the company in our system.
A company can set up a matching gift program and launch giving funds to employees in about 10 minutes of work.
Historically, corporate charitable giving matching programs have been administratively painful to operate. Making payments to charities, maintaining tax records, and doing due diligence on charitable compliance is taxing on HR / finance teams. The necessary software to help has historically been quite expensive and not very useful for employees beyond the matching features.
This is one example of an observation Stephen made after working for years as a philanthropic consultant. Consumer fintech products aren’t built to make great giving experiences for donors. Instead, they are built for buyers — e.g., nonprofits (fundraising) or corporations (gift matching) — without a ton of consideration for the everyday user experience.
A few years back, my wife and I made a commitment to give a portion of our income away every year, and we found it administratively painful to give regularly. The tech that nonprofits typically use hardly inspires generosity — e.g., high fees, poor user flows, and questionable information flow (like tax receipts). Giving platforms try to compensate for poor functionality with bright pictures of happy kids in developing countries, but when the technology is not a good financial experience it puts a damper on things.
Charityvest started when I noticed a particular opportunity with donor-advised funds, which are tax-deductible giving funds recognized by the IRS. They are growing quickly (20% CAGR), but mainly among the high-net worth demographic. We believe they are powerful tools. They enable donors to have a giving portfolio all from one place (on one tax receipt) and have full control over their payment information/frequency, etc. Most of all, they enable a donor to split the decisions of committing to give and supporting a specific organization. Excitement about each of these decisions often strikes at different times for donors—particularly those who desire to give on a budget.
We believe everyone should have their own charitable giving fund no matter their net worth. We’ve created technology that has democratized donor-advised funds.
We also believe good technology should be available for every company, big and small. Employers can offer Charityvest for $2.49 / employee / month subscription, and we charge no fees on any of the giving — charities receive 100% of the money given.
Lastly, we send the program administrator a fun report every month to let them know all the awesome giving their company and its employees did in one dashboard. This info can be leveraged for internal culture or external brand building.
We’re just launching our workplace giving product, but we’ve already built a good portfolio of trusted customers, including Eric Ries’ (author of The Lean Startup) company, LTSE. We’ve particularly seen a number of companies use us as a meaningful part of their corporate decision to join the fight for racial justice in substantive ways.
Our endgame is that the world becomes more generous, starting with the culture of every company. We believe giving is fundamentally good and we want to build technology that encourages more of it by making it more simple and accessible.
You can check out our workplace giving product at (https://charityvest.org/workplace-giving). If you’re interested, we can get your company up and running in 10 minutes. Or, please feel free to forward us on to your HR leadership at your company.
Our giving funds are also available for free for any individual on https://charityvest.org — without gift matching and reporting. We’d invite you to check out the experience. For individuals, we make gifts of cash and stock to any charity fee-free.
Happy to share this with you all, and we’d love to know what you think.
What a great idea!
Are there two separate donations or does it add the company's name after the donor's name? Some way to notify recipients about the low cost of managing a charitable donation match program with your service would be great.
Have you encountered any charitable foundations which prefer to receive cryptoassets? Red Cross and UNICEF accept cryptocurrency donations for the children, for example.
Do you have integration with other onboarding and HR/benefits tools on your roadmap? As a potential employee, I would like to work for a place that matches charitable donations, so mentioning as much in job descriptions would be helpful.
Thanks! Our matching system issues an identical grant from the fund of the matching company. It goes out in the same grant cycle as the employee grant so they go together.
We haven't yet encountered any charity that prefers to receive cryptoassets.
We have lots of dreams about thoughtful integrations with HR software, but we want the experience to be excellent, and we want to keep the experience of our existing app excellent. We'll be balancing those priorities as we grow.
> Our matching system issues an identical grant from the fund of the matching company. It goes out in the same grant cycle as the employee grant so they go together.
So the system creates a separate transaction for the original and the matched donation with each donor's name on the respective gift?
How do users sync which elements of their HR information with your service? IDK what the monthly admin cost there is.
There are a few HR, benefits, contracts, and payroll YC companies with privacy regulation compliance and APIs https://www.ycombinator.com/companies/?query=Payroll
https://founderkit.com/people-and-recruiting/health-insuranc...
Separate data records, yes, separate payment, no. The charity receives one consolidated check in the mail each month across all donors (one payment), and the original donor's grant will be on the data sheet as well as the matching grant from the company's corporate fund (separate records).
Today, users create their accounts directly with our app, and they are affiliated with their corporation in our app via their email address. So we don't integrate with any HR information.
Administrators of the program can add and remove employees via copying and pasting email addresses (can add/remove many at a time). We aim to integrate with HR systems of record in the future to make this seamless.
Thanks for clarifying.
Do you offer a CSV containing donor information to the charity?
Do you support anonymous matched donations?
Can donors specify that a donation is strongly recommended for a specific effort?
...
3% * $1000/yr == $2.50/mo * 12mo
CSV: yes if they request we’ll happily provide.
Anonymous: yes it’s an option on our grant screen
Specific efforts: yes on the grant screen we enable donors to add a “specific need” they’d like their grant to fund.
:-)
It may be helpful to integrate with charity evaluation services to help donors assess various opportunities to give.
Charity Navigator > Evaluation method https://en.wikipedia.org/wiki/Charity_Navigator#Evaluation_m...
We love the idea of layering in rich information to help donors make informed decisions! We just want to be really thoughtful about the experience. So we plan to get there, it just may take us some time.
We Need a Yelp for Doctoral Programs
How are the data needs for such a doctoral and post-doctoral evaluation program different from the data needs for https://collegescorecard.ed.gov ?
Data: https://collegescorecard.ed.gov/data/
Data documentation: https://collegescorecard.ed.gov/data/documentation/
Reviews seem like the biggest need.
“The culture in the X department is terrible, everyone works 80 hours a week and is miserable”
“Night life is great at least, the grad students usually go out for student nights at blah blah after seminar”
“Underrated X program because at university Y, don’t be fooled we literally have 4 of the 5 top researchers in field Z”
Etc.
But it is so hard for people to give reviews. People seem to give good review when are they are very pissed off or very happy. Later seems to be a very rare case.
I have always thought that bad reviews are sufficient for all purposes, and the only reason to even have good reviews is so the reviewees don't feel overly persecuted.
Any product or service will have occasional customers who are either unreasonable or have bad luck. If you have a lot of bad reviews, then as a prospective customer you want to see if all of them fall under the two categories or whether there is a pattern that is relevant to you.
It seems to me that a review database works fine even when all the reviews are bad, and the only regulation it really needs is an effort to prevent one single individual with a vendetta from overwhelming it.
All of the World’s Money and Markets in One Visualization
> Derivatives top the list, estimated at $1 quadrillion or more in notional value according to a variety of unofficial sources.
1 Quadrillion: 1,000,000,000,000,000 (10^15)
Derivative (finance) https://en.wikipedia.org/wiki/Derivative_(finance)
Derivatives market https://en.wikipedia.org/wiki/Derivatives_market :
> The market can be divided into two, that for exchange-traded derivatives and that for over-the-counter derivatives.
Why companies lose their best innovators (2019)
https://news.ycombinator.com/item?id=23886158
> Three reasons companies lose their best innovators.
> 1. They fail to recognize and support the innovators
> 2. Innovation becomes a herculean task
> 3. Corporations don’t match rewards with outcomes
While the paragraphs under point 2 do discuss risk and the paragraphs under point 3 do discuss rewards, I'm not sure this article belongs here.
Risk and Reward.
Large corporations are able to pay people by doing things at scale; with sufficient margin at sufficient volume to justify continued investment. Risk is minimized by focusing on ROI.
Startups assume lots of risk and lots of debt and most don't make it. Liquidation preference applies as the startup team adjourns (and maybe open-sources what remains). In a large corporation, that burnt capital is reported to the board (which represents the shareholders) who aren't "gambling" per se. "You win some and you lose some" is across the street; and they don't have foosball and snacks.
How can large organizations (nonprofit, for-profit, governmental) foster intrapreneurial mindsets without just continuing to say "innovation" more times and expecting things to happen? Drink. "Innovators welcome!". Drink water.
"Intrapreneurial." What does that even mean? The employee, within their specialized department, spends resources (time, money, equipment) on something that their superior managers have not allocated funding for because they want: (a) recognition; (b) job security; (c) to save resources such as time and money; (d) to work on something else instead of this wasteful process; (e) more money.
Very few organizations have anything like "20% time". Why was 20% time thrown off the island to a new island where they had room to run? Do they have foosball? Or is the work so fun that they don't even need foosball? Or is it worth long days and nights because the potential return is enough money to retire tomorrow and then work on what?
Step 1. Steal innovators to work on our one thing
Step 2.
Step 3. Profit.
20% Project: https://en.wikipedia.org/wiki/20%25_Project
Intrapreneurship: https://en.wikipedia.org/wiki/Intrapreneurship
Internal entrepreneur: https://en.wikipedia.org/wiki/Internal_entrepreneur
CINO: Chief Innovation Officer / CTIO: Chief Technology Innovation Officer https://en.wikipedia.org/wiki/Chief_innovation_officer
... Is acquiring innovation and bringing it to scale a top-down process? How do we capture creative solutions and then allocate willing and available resources to making that happen?
awesome-ideation-tools: https://github.com/zazaalaza/awesome-ideation-tools
Powerful AI Can Now Be Trained on a Single Computer
Lots of people are focusing on this being done on a particularly powerful workstation, but the computer described seems to have power at a similar order of magnitude to the many servers which would be clustered together in a more traditional large ML computation. Either those industrial research departments could massively cut costs/increase output by just “magically keeping things in ram,” or these researchers have actually found a way to reduce the computational power that is necessary.
I find the efforts of modern academics to do ML research on relatively underpowered hardware by being more clever about it to be reminiscent of soviet researchers who, lacking anything like the access to computation of their American counterparts, were forced to be much more thorough and clever in their analysis of problems in the hope of making them tractable.
If anything it seems to me that doing the most work under constraint of resources is precisely what intelligence is about. I've always wondered why the consumption of compute resources is itself not treated as a significant part of the 'reward' in ML tasks.
At least if you're taking inspiration from biological systems, it clearly is part of the equation, a really important one even.
Ask HN: Something like Khan Academy but full curriculum for grade schoolers?
Khan Academy continually gets held up as a great resource for online courses across the age spectrum for math related subjects. With the continuing pandemic continuing to grow in the US and schools not really sure how to handle things, the GF and I are looking into other options.
Is there a recommended resource that gives unbiased (as possible) reviews for middle school (7-8th grade) curriculum? Searching these days really doesn't bring up quality, just options one has to comb through.
K12 CS Framework (ACM, Code.org, [...]) https://k12cs.org/
Computing Curricula 2020 (ACM, IEEE,) http://www.cc2020.net/
Official SAT Practice (College Board, Khan Academy) https://www.khanacademy.org/sat
http://wrdrd.github.io/docs/consulting/software-development#... (TODO: add link to cc2020 draft)
Programmer Competency Matrix: http://sijinjoseph.com/programmer-competency-matrix/ , https://competency-checklist.appspot.com/ , https://github.com/hltbra/programmer-competency-checklist
Re: Computational thinking https://westurner.github.io/hnlog/#comment-15454421
Coding Interview University: https://github.com/jwasham/coding-interview-university
AutoML-Zero: Evolving Code That Learns
They use evolutionary search [0] to create programs for image classification, specifically 'binary classification tasks extracted from CIFAR-10'. They do it from scratch, though they use a pytorch-ish programming language with differentiation baked in.
Looks like a nice summer intern project.
"AutoML-Zero: Evolving Machine Learning Algorithms From Scratch" (2020) https://arxiv.org/abs/2003.03384 https://scholar.google.com/scholar?cluster=11748751662887361...
How does this compare to MOSES (OpenCog/asmoses) or PLN? https://github.com/opencog/asmoses https://scholar.google.com/scholar?hl=en&as_sdt=0%2C43&q=%22... (2007)
SymPy - a Python library for symbolic mathematics
SymPy is the best!
In addition to the already-excellent official tutorial https://docs.sympy.org/latest/tutorial/index.html , I have written a short printable tutorial summary of the functions most useful for students https://minireference.com/static/tutorials/sympy_tutorial.pd...
For anyone interested in trying SymPy without installing, there is also the online shell https://live.sympy.org/ which works great and allows you to send entire computations as a URL, see for example this comment that links to an important linear algebra calculation: https://news.ycombinator.com/item?id=23158095
How does it compare to Matlab's symbolic toolbox, aside from not being affordable outside of academia?
NumPy for Matlab users: https://numpy.org/doc/stable/user/numpy-for-matlab-users.htm...
SymPy vs Matlab: https://github.com/sympy/sympy/wiki/SymPy-vs.-Matlab
If you then or later need to do distributed ML, it is advantageous to be working in Python. Dask Distributed, Dask-ML, RAPIDS.ai (CuDF), PyArrow, xeus-cling
Good to know: sympy does not use numpy so you can use pypy instead of python.
In my case I got a 7x speedup.
You can however use sympy to convert symbolic expressions to numeric numpy functions, using the "lambdify" feature. It's awesome because it lets you symbolically generate and manipulate a numerical program.
SymEngine https://github.com/symengine/symengine
> SymEngine is a standalone fast C++ symbolic manipulation library. Optional thin wrappers allow usage of the library from other languages, e.g.:
> [...] Python wrappers allow easy usage from Python and integration with SymPy and Sage (the symengine.py repository)
https://en.wikipedia.org/wiki/SymPy > Related Projects:
> SymEngine: a rewriting of SymPy's core in C++, in order to increase its performance. Work is currently in progress to make SymEngine the underlying engine of Sage too
How does it compare to sage math? I used sage math today and was pleasantly surprised. Even has a latex option. If you name your variables correctly, it even outputs Greek symbols and subscript correctly!
User 29athrowaway mentions in another comment that they use Sympy through Sagemath, so they probably could answer your question best. It sounds like Sympy is a subset of what Sage offers, but I'm not familiar enough with either tool to know.
Ask HN: Are there any messaging apps supporting Markdown?
I'd like to easily send formatted code, and bullet points, etc. through a messaging app without having to resort to a heavy app like Slack.
Mattermost supports CommonMark Markdown: https://docs.mattermost.com/help/messaging/formatting-text.h...
Zulip supports ~CommonMark Markdown: https://zulip.readthedocs.io/en/latest/subsystems/markdown.h...
Reddit supports Markdown. https://www.reddit.com/wiki/markdown
Discourse now supports CommonMark Markdown.
GitHub, BitBucket, GitLab and Gogs/Gitea support Markdown.
I wouldn't consider Discourse/Reddit/Github/etc. to be a messaging app per se, even if some of those have messaging functionality between users...
I digress on the category definition. Public messaging (without PM or DM features) is still messaging; and often far more useful than trying to forward 1:1 messages in order to bring additional participants onboard.
It's worth noting that GH/BB/GL have all foregone PM features; probably for the better in terms of productivity: messaging @all is likely more productive.
What vertical farming and ag startups don't understand about agriculture
My father is an ag soil chemist of 50+ years.
I'm an industrial systems eng. w/ a specialty in polymer-textile-fiber engineering. (Mostly useless skillsets in the US now)
Gonna share a few lessons here about agriculture that I try to convey to EECS, econ, Neuroscience, and the web developer crowd.
- You can only grow non-calorically dense foods in vertical farms
- It takes 10-14 kwh/1000 gallons of water to desalinate. More if it gets periodically polluted at an increasing rate.
- Large majority Agrarian populations exist because the countries are stuck in a purgatory of <1 MWh/capita annum whereby the country doesn't have scaleable nitrogen and steel manufacturing.
- Sweet potatoes and sweet potatoes are some of the highest satiety lowest input to output ratio produce. High efficiency.
- In civilizations where you are at < 1MWh/capita annum - there is not enough electricity to produce tools for farming, steel for roads, and concrete for building things. The end result is that the optimal decision is to have more children to harvest more calories per an acre.
- Property, bankruptcy, and inheritance law have an immense influence on the farmer population of a country.
I remember telling some "ag tech" VCs my insights and offering to introduce my father who has an immense amount of insight on the topic from having grown things for as long as he has....My thoughts were tossed aside.
So you can't grow potatoes vertically? Can you elaborate? Is it a function of physiology, i.e. calorie dense vegetables need far more leaves and supporting stems than can be practically stacked vertically?
I imagine space is a factor, but energy will be a big one as well. Calorie dense foods will likely need more space and energy (light) inputs. Vertical farms are very water efficient, so I don't think that matters much.
Vertical farms make a lot more sense with fresh vegetables like leafy greens that grow quickly, command high prices if grown organically, and benefit from being closer to market.
Potatoes are the exact opposite. If it ever becomes more cost effective to grow corn, wheat, and potatoes in virtual farms then outdoor agriculture is dead. While I don't agree with the article that it will never happen, it might require energy advances like fusion power or drastically higher _rural_ land values and water prices.
Greenhouses make sense long before vertical farming, just look at agriculture in the Netherlands, it's mind boggling how much they produce for such a tiny country.
Can you expand on this?
I get that to store a calorie in a potato I need to supply a calorie of energy from somewhere else.
But why is fusion power required instead of better UV lamps in my vertical farm? (Assuming I had enough electricity to run them)
The total amount of electricity to power those UV lamps should be on par with what the Sun sends to the potatoes fields. Maybe that's the reason for fusion. It didn't do the math.
Actually no not really. Plants only absorb two wavelengths of light. It's currently more efficient to convert sun into solar power via panels and then to light LEDs supplying only the wavelengths that plants use. Despite the seeming inefficiency here, the fact is that plants are even more inefficient at absorbing light not at the right wavelengths than solar panels.
Could one imagine a material that would absorb solar spectrum and emit the preferred frequencies? Something like a polymer one could stretch over fields to get more from the suns rays.
>> "Actually no not really. Plants only absorb two wavelengths of light. It's currently more efficient to convert sun into solar power via panels and then to light LEDs supplying only the wavelengths that plants use. Despite the seeming inefficiency here, the fact is that plants are even more inefficient at absorbing light not at the right wavelengths than solar panels."
> Could one imagine a material that would absorb solar spectrum and emit the preferred frequencies? Something like a polymer one could stretch over fields to get more from the suns rays.
Would you call that a "solar transmitter"?
https://en.wikipedia.org/wiki/Transmitter :
> Generators of radio waves for heating or industrial purposes, such as microwave ovens or diathermy equipment, are not usually called transmitters, even though they often have similar circuits.
Would "absorption spectroscopy" specialists have insight into whether this is possible without solar cells, energy storage, and UV LEDs? https://en.wikipedia.org/wiki/Absorption_spectroscopy
(edit) The thermal energy from sunlight (from the FREE radiation from the nuclear reaction at the center of our solar system) is also useful to and necessary for plants. There's probably a passive heat pipe / solar panel cooling solution that could harvest such heat for colder seasons and climates.
Also, UV-C is useful for sanitizing (UVGI) but not really for plant growth. https://en.wikipedia.org/wiki/Ultraviolet_germicidal_irradia... :
> UVGI can be coupled with a filtration system to sanitize air and water.
Is that necessary or desirable for plants?
https://www.lumigrow.com/learning-center/blogs/the-definitiv... :
> The light that plants predominately use for photosynthesis ranges from 400–700 nm. This range is referred to as Photosynthetically Active Radiation (PAR) and includes red, blue and green wavebands. Photomorphogenesis occurs in a wider range from approximately 260–780 nm and includes UV and far-red radiation.
Photomorphogenesis: https://en.wikipedia.org/wiki/Photomorphogenesis
PAR: Photosynthetically active radiation: https://en.wikipedia.org/wiki/Photosynthetically_active_radi...
Grow light: https://en.wikipedia.org/wiki/Grow_light
Are there bioluminescent e.g. algae which emit PAR and/or UV? Algae can feed off of waste industrial gases.
Bioluminescence > Light production: https://en.wikipedia.org/wiki/Bioluminescence#Light_producti...
Biophoton: https://en.wikipedia.org/wiki/Biophoton
Chemiluminescence: https://en.wikipedia.org/wiki/Chemiluminescence
Electrochemiluminescence: https://en.wikipedia.org/wiki/Electrochemiluminescence
Quantum dot display / "QLED": https://en.wikipedia.org/wiki/Quantum_dot_display
Could be possible? Analyzing the inputs and outputs is useful in natural systems, as well.
Ask HN: What are your go to SaaS products for startups/MVPs?
Looking for some inspiration. Ive done a lot of MVPs/Early-stage apps over the years and I tend to lean on the same SaaS portfolio for mails, text gateways, payment etc, but Im sure Ive missed a few valuable additions.
Here's a few I use: Mails: Mailchimp / Mandrill Payment: Paylike Search: Algolia
https://StackShare.io and https://FounderKit.com are great places to find reviews of SaaS services:
> mails,
https://founderkit.com/growth-marketing/email-marketing/revi...
https://stackshare.io/email-marketing
https://zapier.com/learn/email-marketing/
> text gateways,
https://founderkit.com/apis/sms/reviews
https://stackshare.io/voice-and-sms
> payments
https://founderkit.com/apis/credit-card-processing/reviews
https://stackshare.io/payment-services
Both have categories:
https://stackshare.io/categories
https://founderkit.com/reviews
If you're looking for market/competition research, I'd recommend https://coscout.com as well, for info about companies, funding, competitors, tech stack etc
It's in alpha right now, so you'll have to get in the waitlist, though they're only a few weeks away from launching AFAIK.
Full disclosure: A friend of mine is building this :)
Long term viability of SaaS solutions is definitely worth researching.
Is this something that's going to get acquired and be extinguished?
What are our switching costs?
How do we get our data in a format that can be: read into our data warehouse/lake and imported into an alternate service if necessary in the future?
How does Coscout compare to e.g. Crunchbase, PitchBook (Morningstar), YCharts, AngelList?
Ask HN: Do you read aloud or silently in your minds?
Most times while reading a new topic that I am not familiar with, I tend to read aloud in my mind. Yet that changes based on the content and the way it is written.
When I'm focused, I notice that reading silently helps increase my reading speed and cognition, like everything is flowing in.
Other times I don't seem to understand anything if I'm not reading it aloud in my mind.
Has anyone noticed such a thing and if so can you share any tips or information you've learned about this behavior
Subvocalization https://en.wikipedia.org/wiki/Subvocalization
Speed reading https://en.wikipedia.org/wiki/Speed_reading
Ask HN: How do you deploy a Django app in 2020?
Hi. I'm a mid-level software engineer trying to deploy a small (2000 max users) django app to production.
If I Google: How to deploy a Django app. I get 10+ different answers.
Can anyone on HN help me please.
You'll probably get 10 different answers here too.
I use uwsgi in emperor mode. I have a Makefile that runs a few SSH commands on production to clone my git repo at a specific commit, builds a virtual environment from requirements.txt, and atomically swaps a symlink to a uwsgi socket. Nginx in front of it all. It's pretty much the Python equivalent of FTPing HTML files to a web server.
Uwsgi is terrifying because it has so many options (and I wonder how secure it is), but it has never failed me. Packaging and containerizing things sounds cool, but I just can't justify spending time on it when my setup works fine (I'm a solo dev).
If you have only one production server, dokku is "A docker-powered PaaS that helps you build and manage the lifecycle of applications." Dokku supports Heroku buildpack deployment (buildstep), Procfiles, Dockerfile deployment, Docker image deployment, git deployment (gitreceive), or tarfile deployments. https://github.com/dokku/dokku
There are a number of plugins for Dokku. Dokku ships with the nginx plugin as the HTTP frontend proxy. Dokku supports SSL certs with the certs plugin.
When you need to move to more than one server, what do you do? There's now a dokku-scheduler-kubernetes plugin which can do HA (high availability) which is worth reading about before you develop and document your own deployment workflow. https://github.com/dokku/dokku-scheduler-kubernetes
I also always put build, test, and deployment commands in a Makefile.
Package it; as a container or as containers that install a RPM/DEB/APK/Condapkg/Pythonpkg (possibly containing a zipapp). Zipapps are fast.
If you have any non-python dependencies, a Pythonpkg only solves for part of the packaging needs.
Producing a packaged artifact should be easy and part of your CI build script.
Here's the cookiecutter-django production docker-compose.yml with containers for django, celery, postgres, redis, and traefik as a load balancer: https://github.com/pydanny/cookiecutter-django/blob/master/%...
Cookiecutter-django also includes a Procfile.
With k8s, you have an ingress (~load balancer + SSL termination proxy) other than traefik.
You can generate k8s YML from docker-compose.yml with Kompose.
I just found this which describes using GitLab CI with Helm: https://davidmburke.com/2020/01/24/deploy-django-with-helm-t...
What is the command to scale up or down? Do you need a geodistributed setup (on multiple providers' clouds)? Who has those credentials and experience?
How do you do red/green or rolling deployments?
Can you run tests in a copy of production?
Can you deploy when the tests that run on git commit pass?
What runs the database migrations in production; while users are using the site?
If something deletes the whole production setup or the bus factor is 1, how long does it take to redeploy from zero; and how much manual work does it take?
CI + Ansible + Terraform + Kubernetes.
Whatever tools you settle on, django-eviron for a 12 Factor App may be advisable. https://github.com/joke2k/django-environ
The Twelve-Factor App: https://12factor.net/
Containers from first principles
I recently discovered systemd-nspawn and was amazed at how lightweight a basic container can be.
"Docker Without Docker" (2015) explains /sbin/init and systemd-nspawn. Systemd did not exist when docker was first created. https://chimeracoder.github.io/docker-without-docker/
That's not what I remember. Wikipedia backs up that recollection as well [0], marking systemd's initial release in 2010. Docker's initial release is listed as 2013 [1]. Maybe that was true of the dotCloud internal releases, and certainly, not all of the EL and other Linux distros had not adopted systemd during 2010 - 2013. Certainly after 2014 or 2015, systemd had spread to the major Linux distros so Docker could have chosen to take a systemd-based approach at that point.
Are there other systemd + containers solutions?
"Chapter 4. Running containers as Systemd services with Podmam" https://access.redhat.com/documentation/en-us/red_hat_enterp...
AFAIU, when running containers with systemd:
- logs go to journald by default
- there's no docker-compose for just the [name-prefixed] containers in the docker-compose.yml,
- you can use systemd unit template parametrization
- it's not as easy to collect metrics on every container on the system without a read-only docker socket: how many containers are running, how much RAM quota are they assigned and utilizing? What are the filesystem and port mappings?
- you can run containers as non-root
- you can run containers in systemd timer units
- you use runC to handle seccomp
... You can do cgroups and namespaces with just systemd; but keeping chroots/images upgraded is outside the scope of systemd: where is the ideal boundary between systemd and containers?
See this comment regarding per-container MAC MCS labels: https://news.ycombinator.com/item?id=23430959
There's much additional complexity that justifies k8s / OpenShift: when would I want to manage containers with just systemd units?
> Many people might think the word “container” has a specific meaning within the Linux kernel; however the kernel has no notion of a “container”. The word has been synonymous with a variety of Linux tooling which when applied give the resemblance of what we expect a container to be.
Before LXC ( https://LinuxContainers.org ) and CNCF ( https://landscape.cncf.io/ ) and OCI ( https://opencontainers.org/ ), for shared-kernel VPS hosting ("virtual private server"; root on a shared box), there was OpenVZ (which requires a patched kernel and AFAIU still has features, like bursting, not present in cgroups).
Docker no longer has an LXC driver: libcontainer (opencontainers/runc) is the story now. The LXC docs have a great list of utilized kernel features that's also still true for docker-engine = runC + moby. The LXC docs: https://linuxcontainers.org/lxc/introduction/ :
> Current LXC uses the following kernel features to contain processes:
> ## Kernel namespaces (ipc, uts, mount, pid, network and user)
>> Namespaces are a feature of the Linux kernel that partitions kernel resources such that one set of processes sees one set of resources while another set of processes sees a different set of resources. https://en.wikipedia.org/wiki/Linux_namespaces
> ## Apparmor and SELinux profiles https://en.wikipedia.org/wiki/AppArmor / https://en.wikipedia.org/wiki/Security-Enhanced_Linux
udica is an interesting tool for creating SELinux policies for containers.
Is it possible for each container to run confined with a different SELinux label?
> ## Seccomp policies https://en.wikipedia.org/wiki/Seccomp
See below re: Seccomp.
> ## Chroots (using pivot_root) https://en.wikipedia.org/wiki/Chroot
Chroots and symlinks, Chroots and bind mounts, Chroots and overlay filesystems, Chroots and SELinux context labels.
FWIU, Chroots are a native feature of filesystem syscalls in Fuchsia.
> ## Kernel capabilities
https://wiki.archlinux.org/index.php/Capabilities :
>> "Capabilities (POSIX 1003.1e, capabilities(7)) provide fine-grained control over superuser permissions, allowing use of the root user to be avoided. Software developers are encouraged to replace uses of the powerful setuid attribute in a system binary with a more minimal set of capabilities. Many packages make use of capabilities, such as CAP_NET_RAW being used for the ping binary provided by iputils. This enables e.g. ping to be run by a normal user (as with the setuid method), while at the same time limiting the security consequences of a potential vulnerability in ping."
> ## CGroups (control groups)* https://en.wikipedia.org/wiki/Cgroups
Control groups enable per-process (and to thus per-container) resource quotas. Other than limiting the impact of resource exhaustion, cgroups are not a security feature of the Linux kernel.
Here's a helpful explainer of the differences between some of these kernel features; which, combined, have become somewhat ubiquitous:
From "Formally add support for SELinux" (k3s #1372) https://github.com/rancher/k3s/issues/1372#issuecomment-5817... :
> https://blog.openshift.com/securing-kubernetes/*
>> The main thing to understand about SELinux integration with OpenShift is that, by default, OpenShift runs each container as a random uid and is isolated with SELinux MCS labels. The easiest way of thinking about MCS labels is they are a dynamic way of getting SELinux separation without having to create policy files and run restorecon.*
>> If you are wondering why we need SELinux and namespaces at the same time, the way I view it is namespaces provide the nice abstraction but are not designed from a security first perspective. SELinux is the brick wall that’s going to stop you if you manage to break out of (accidentally or on purpose) from the namespace abstraction.
>> CGroups is the remaining piece of the puzzle. Its primary purpose isn’t security, but I list it because it regulates that different containers stay within their allotted space for compute resources (cpu, memory, I/O). So without cgroups, you can’t be confident your application won’t be stomped on by another application on the same node.
From Wikipedia: https://en.wikipedia.org/wiki/Seccomp ::
> seccomp (short for secure computing mode) is a computer security facility in the Linux kernel. seccomp allows a process to make a one-way transition into a "secure" state where it cannot make any system calls except exit(), sigreturn(), read() and write() to already-open file descriptors. Should it attempt any other system calls, the kernel will terminate the process with SIGKILL or SIGSYS.[1][2] In this sense, it does not virtualize the system's resources but isolates the process from them entirely.
... SELinux is one implementation of MAC (Mandatory Access Controls) that is built upon the LSM (Linux Security Modules) support in the Linux kernel. Some distros include policy sets for Docker hosts and lots of other packages that could be installed; see: "Formally add support for SELinux" (k3s #1372) https://github.com/rancher/k3s/issues/1372#issuecomment-5817...
How many people did it take to build the Great Pyramid?
> The potential energy of the pyramid—the energy needed to lift the mass above ground level—is simply the product of acceleration due to gravity, mass, and the center of mass, which in a pyramid is one-quarter of its height. The mass cannot be pinpointed because it depends on the specific densities of the Tura limestone and mortar that were used to build the structure; I am assuming a mean of 2.6 metric tons per cubic meter, hence a total mass of about 6.75 million metric tons. That means the pyramid’s potential energy is about 2.4 trillion joules.
In "Lost Technologies of the Great Pyramid" (2010) and "The Great Pyramid Prosperity Machine: Why the Great Pyramid was Built!" (2011), Steven Myers contends that the people who built the pyramids were master hydrologists who built a series of locks from the Nile all the way up the sides of the pyramids and pumped water up to a pool of water on the topmost level; where they used buoyancy and mechanical leverage by way of a floating barge crane in order to place blocks. This would explain how and why the pyramids are water tight, why explosive residue has been found in specific chambers, and why boats have been found buried at the bases of the pyramids.
https://www.amazon.com/dp/B0045Y26CC/
There are videos: http://www.thepump.org/video-series-2
https://www.youtube.com/playlist?list=PLt_DvKGJ_QLYvJ3IdVKXU...
I'm not aware of other explanations for how friction could have been overcome in setting the blocks such that they are watertight (in the later Egyptian pyramids).
AFAIU, the pyramids of South America appear to be of different - possibly older - construction methods.
Solar’s Future is Insanely Cheap
Power markets are more complicated than most people realize. One thing to note is that solar power can be cheaper than gas, but still not be economic. The fact of the matter is that an intermittent kWh is not as valuable as an on-demand reliable kWh to a utility who's number 1 priority is reliability. Even as solar is acquired at lower and lower prices, if evening power is generated by expensive and inefficient gas turbines, the customer might not see costs go down (and emissions might not go down either!). Solar is clearly economic in many markets, but we'll never get grid emissions in a place like California down much more without storage.
As mentioned in the article, solar power is insanely cheap. So while you are right that a kWh of solar is not, on average, as valuable as a kWh of semidispatchable coal, the insane cheapness means that at some point, it is economical to massively overbuild renewable and decommission fossil fuel plants. You provide no evidence for the implausible claim that solar causes emissions to go up: your argument is only that the emissions reducing effect is muted, which may be true.
Also, while storage would be helpful, it is not the only way to enable a renewable transition. Additional transmission is enormously helpful: as the sun goes down on California, the wind is picking up in the Midwest. And don’t forget demand response: if smart thermostats received price signals (maybe we should precool this house...) that would alleviate the evening ramp-up issue.
So I claim we’ll need less storage than “a whole day’s usage”. But the learning curve applies to batteries as well! This storage won’t cost as much anyway.
The whole issue of intermittency is overrated. While a single solar panel might generate intermittently, the solar fleet across a whole state generates more predictably.
I am predicting that grid emissions will come down a lot over the coming decades. Partially I’m predicting the past: they’ve already come down, a lot!
> if smart thermostats received price signals (maybe we should precool this house...) that would alleviate the evening ramp-up issue.
Is there an existing model for retail intraday rates? Would intraday rates be desirable for all market participants?
"Add area for curtailment data?" https://github.com/tmrowco/electricitymap-contrib/issues/236...
Demo of an OpenAI language model applied to code generation [video]
So that's basically program synthesis from natural language (ish) specifications (i.e. the comments).
I can see this being a useful tool [1]. However, I don't expect any ability for innovation. At best this is like having an exceptionally smart autocomplete function that can look up code snippets on SO for you (provided those code snippets are no longer than one line).
That's not to say that it can't write new code, that nobody has quite written before in the same way. But in order for a tool like this to be useful it must stick as close as possible to what is expected- or it will slow development down rather than helping it. Which means it can only do what has already been done before.
For instance- don't expect this to come up with a new sorting algorithm, out of the blue, or to be able to write good code to solve a certain problem when the majority of code solving that problem on github happens to be pretty bad.
In other words: everyone can relax. This will not take your job. Or mine.
____________
[1] I apologise to the people who know me and who will now be falling off their chairs. OK down there?
I think you are underselling the potential of a model which deeply understand programming. Imagine combining such a model with something like AutoML-Zero: https://arxiv.org/abs/2003.03384 It may not be 'creative', but used as tab-completion, it's not being rewarded or incentivized or used in any way which would expose its abilities towards creating a new sort algorithm.
I agree on the tab-completion part. Something like Gmail's smart-compose could have potentially huge benefits here.
But I'm not sure about the "deeply understand programming" part. Language modelling and "AI", in its current form, uncovers only statistical correlations and barely scratches the surface of what "understanding" is. This has restricted deployment of majority of academic research into the real-world and this, I believe, is no different and will work only in constrained settings.
Edit: typo
It would be nice to have an AI that could write unit tests, or look over your code and understand and explain where you might have bugs.
> look over your code and understand and explain where you might have bugs.
This would certainly be interesting. I'm not aware of active research going on in this area (any pointers would be helpful!).
This would require an agent to have thorough understanding of the logic you're trying to implement, and locate the piece of code where it silently fails. For this you'd again need a training dataset where the input is a piece of code and the supervision signal (the output) is location of the bug. I could imagine some sort of self-supervision to tackle this initially where you'd intentionally introduce bugs in your code to generate training data. But not sure how far this can go!
1. Generate test cases from function/class/method definitions.
2. Generate test cases from fuzz results.
3. Run tests and walk outward from symbols around relevant stacktrace frames (line numbers,).
4. Mutate and run the test again.
...
Model-based Testing (MBT) https://en.wikipedia.org/wiki/Model-based_testing
> Models can also be constructed from completed systems
> At best this is like having an exceptionally smart autocomplete function that can look up code snippets on SO for you (provided those code snippets are no longer than one line).
Yeah, all it could do for you is autocomplete around what it thinks the specification might be at that point in time.
> But what if Andy gets another dinosaur, a mean one? -- Toy Story (1995)
Future of the human climate niche
How many degrees Celsius hotter would that be for billions of people in 50 years?
> The Paris Agreement's long-term temperature goal is to keep the increase in global average temperature to well below 2 °C above pre-industrial levels; and to pursue efforts to limit the increase to 1.5 °C, recognizing that this would substantially reduce the risks and impacts of climate change. This should be done by reducing emissions as soon as possible, in order to "achieve a balance between anthropogenic emissions by sources and removals by sinks of greenhouse gases" in the second half of the 21st century. It also aims to increase the ability of parties to adapt to the adverse impacts of climate change, and make "finance flows consistent with a pathway towards low greenhouse gas emissions and climate-resilient development."
> Under the Paris Agreement, each country must determine, plan, and regularly report on the contribution that it undertakes to mitigate global warming. [6] No mechanism forces [7] a country to set a specific emissions target by a specific date, [8] but each target should go beyond previously set targets.
And then this is what was decided:
> In June 2017, U.S. President Donald Trump announced his intention to withdraw the United States from the agreement. Under the agreement, the earliest effective date of withdrawal for the U.S. is November 2020, shortly before the end of President Trump's 2016 term. In practice, changes in United States policy that are contrary to the Paris Agreement have already been put in place.[9]
https://en.wikipedia.org/wiki/Paris_Agreement
Ask HN: Best resources for non-technical founders to understand hacker mindset?
Background: technical founder wondering what reading to recommend to a business focused founder for them to grok the hacker mindset. I've thought perhaps Mythical Man Month and How To Become A Hacker (Eric Raymond essay) but not sure they're quite right.
Any suggestions?
(In case it helps an analogue in the mathematical world might be A Mathematician's Apology or Gödel, Escher, Bach.)
Let me sum up all possible books about understanding the "hacker" (terrible word by the way, because of multiple meanings, which meaning are we talking about?) mindset, to a management perspective:
1) True "hackers" value knowledge over money.
2) True "hackers" value doing things once and doing them right, no matter how long that takes. (Compare to the business mindset of "we need it now", or "we needed it yesterday")
3) True "hackers" value taking ownership in their work, that is, whatever they work on becomes an extension of themselves, much like an artist working on a work of art.
4) True "hackers" are not about work-arounds. If/when work-arounds are used, it's because there there's an artificial timeframe (as might be found in the corporate world), and there's a lack of understanding in the infrastructure which created the need for that work-around.
But, all of these virtues run counter to the demands of business, which constantly wants more things done faster, cheaper, with more features, more complexity, less testing, and doesn't want to worry about problems that may be caused by all of those things in the future (less accountability) -- as long as customer revenue can be collected today.
You see, a true "hacker's" values -- are completely different than those of big business...
And business people wonder why there's stress and burnout among tech people...
> 3) True "hackers" value taking ownership in their work, that is, whatever they work on becomes an extension of themselves, much like an artist working on a work of art
There's something to be said about owning your work, but I have to disagree that unhealthy attachment to work products is a universal attribute of technical founder hackers. It's not a kid, it's a thing that was supposed to be the best use of the resources and information available at the time.
I must have confused this point with vanity and retention in projecting my own counterproductive anti-patterns.
Prolific is not the objective for a true hacker, but not me but a guy I know mentioned something about starting projects and seeing the next 5 years of potentially happily working on that project, too.
Dissecting the code responsible for the Bitcoin halving
> The difficulty of the calculations are determined by how many zeroes need to be at the front. [...]
The difficulty is actually not determined by the number of zeroes (as was initially the case).
https://en.bitcoinwiki.org/wiki/Difficulty_in_Mining :
> The Bitcoin network has a global block difficulty. Valid blocks must have a hash below this target. Mining pools also have a pool-specific share difficulty setting a lower limit for shares.
"Less than" instead of "count leading zeroes" makes it possible for the difficulty to be less broadly adjusted in a difficulty retargeting.
Difficulty retargetings occur after up to 2016 blocks (~10 minutes, assuming the mining pool doesn't suddenly disappear resulting in longer block times that could make it take months to get to 2016 blocks according to "What would happen if 90% of the Bitcoin miners suddenly stopped mining?" https://bitcoin.stackexchange.com/questions/22308/what-would... )
Difficulty is adjusted up or down (every up to 2016 blocks) in order to keep the block time to ~10 minutes.
The block reward halving occurs every ~4 years (210,000 blocks).
Relatedly, Moore's law observes/predicts that processing power (as measured by transistor count per unit) will double every 2 years while price stays the same. Is energy efficiency independent of transistor count? https://en.wikipedia.org/wiki/Moore%27s_law
Ask HN: Does mounting servers parallel with the temperature gradient trap heat?
Heat rises. Is heat trapped in the rack? Would mounting servers sideways (vertically) allow heat to transfer out of the rack?
Many systems have taken the vertical mount approach approach over the years: Blade servers, routers, modems, and various gaming systems.
Horizontally-mounted: parallel with the floor
Vertically-mounted: perpendicular to the floor
[deleted]
Thermodynamics https://en.wikipedia.org/wiki/Thermodynamics
Are engine cylinders ever mounted horizontally? Why or why not?
> Heat rises.
Warmer air is less dense / more buoyant; so it floats.
"Does hot air really rise?" https://physics.stackexchange.com/questions/6329/does-hot-ai...
- Water ice floats because – somewhat uniquely – solid water is less dense than liquid water.
> Is heat trapped in the rack?
Probably.
> Would mounting servers sideways (vertically) allow heat to transfer out of the rack?
How could we find studies that have already tested this hypothesis?
What does the 'rc' in `.bashrc`, etc. mean?
For me, the hard part is remembering if .bashrc is supposed to source .bash_profile or vice versa.
info bash -n "Bash Startup Files"
Google ditched tipping feature for donating money to sites
> When asked, Google confirmed that the designs were an internal idea it explored last year but decided not to pursue as part of [Google Contributor] and Google Funding Choices, which lets sites ask visitors to disable ad blockers, or instead buy a subscription or pay a per page fee to remove ads.
Could this be built on Web Monetization API (ILP (Interledger Protocol)) and e.g. Google Pay as one of many possible payment/card/cryptocurrency processing backends; just like Coil is built on Web Monetization API?
Innovating on Web Monetization: Coil and Firefox Reality
Coil: $5/mo, Content creators get a proportional cut of that amount according to what is browsed with the browser extension enabled or the Puma browser, and Private: No Tracking
> Coil sends payments via the Interledger Protocol, which allows any currency to be used for sending and receiving.
It looks like the Web Monetization API is not yet listed on the Website Monetization Wikipedia page: https://en.wikipedia.org/wiki/Website_monetization
Quoting from earlier this week:
> Web Monetization API (ILP: Interledger Protocol)
>> A JavaScript browser API which allows the creation of a payment stream from the user agent to the website.
>> Web Monetization is being proposed as a #W3C standard at the Web Platform Incubator Community Group.
> https://webmonetization.org/
> Interledger: Web Monetization API https://interledger.org/rfcs/0028-web-monetization/
Ask HN: Recommendations for online essay grading systems?
Which automated essay grading systems would you recommend? Are they open source?
How can we identify biases in these objective systems?
What are your experiences with these systems as authors and graders?
Who else remembers using the Flesch-Kincaid Grade Level metric in Word to evaluate school essays? https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readabi...
Imagine my surprise when I learned that this metric is not one that was created for authors to maximize: reading ease for the widest audience is not an objective in some deparments, but a requirement.
What metrics do and should online essay grading systems present? As continuous feedback to authors, or as final judgement?
I'm reminded of a time in highschool when an essay that I wrote was flagged by an automated essay verification engine as plagiarism. I certainly hadn't plagiarized, and it was up to me to refute that each identified keyword-similar internet resource on the internet was not an uncited source of my paper. I disengaged. I later wrote an essay about how keyword search tools could be helpful to students doing original research. True story.
Decades later, I would guess that human review is still advisable.
This need of mine to have others validate my unpaid work has nothing to do with that traumatic experience.
I still harbor this belief in myself: that what I have to say is worth money to others, and that - someday - I'll pay a journal to consider my ScholarlyArticle for publishing in their prestigious publication with maybe even threaded peer review (and #StructuredPremises linking to Datasets and CreativeWorks that my #LinkedMetaAnalyses are predicated upon). Someday, I'll develop an online persona as a scholar, as a teacher, maybe someday as a TA or an associate professor and connect my CV to any or all of the social networks for academics. I'll work to minimize the costs of interviewing and searching public records. My research will be valued and funded.
Or maybe, like 20% time, I'll find time and money on the side for such worthwhile investigations; and what I produce will be of value to others: more than just an exercise in hearing myself speak.
In my years of internet communications, I've encountered quite a few patrons; lurkers; participants; and ne'er-do-wells who'll order 5 free waters, plaster their posters to the walls, harass paying customers, and just walk out like nothing's going to happen. Moderation costs time and money; and it's a dirty job that sometimes pays okay. There are various systems for grading these comments, these essays, these NewsArticles, these ScholarlyArticles. Human review is still advisable.
> How can we identify biases in these objective systems?
Modern "journalism" recognizes that it's not a one-way monologue but a dialogue: people want to comment. Ignorantly, helpfully, relevantly, insightfully, experiencedly. What separates the "article part" from the "comments part" of the dialogue? Typesetting, CSS, citations, quality of argumentation?
Ask HN: Systems for supporting Evidence-Based Policy?
What tools and services would you recommend for evidence-based policy tasks like meta-analysis, solution criteria development, and planned evaluations according to the given criteria?
Are they open source? Do they work with linked open data?
> Ask HN: Systems for supporting Evidence-Based Policy?
> What tools and services would you recommend for evidence-based policy tasks like meta-analysis, solution criteria development, and planned evaluations according to the given criteria?
> Are they open source? Do they work with linked open data?
I suppose I should clarify that citizens, consumers, voters, and journalists are not acceptable answers
Facebook, Google to be forced to share ad revenue with Australian media
What if Google and Facebook just said “No, thank you.”? I think Google and Facebook have more power in this relationship, which might say something about the tech giants. The only possible thing the Aussies could do is block the websites, and that sounds like political suicide to me. I’m guessing the companies will decide to play ball, but if other countries start to follow suit, it might be interesting to see what happens if they start pushing back.
There would be no political suicide there. It's so easy to make Google/Facebook look evil in this case if they don't comply. Just say they are not paying taxes and getting our money abroad etc.
But there is 0% chance of Google/Facebook not complying.
> I think Google and Facebook have more power in this relationship
Not even close.
Don't the French and Spanish examples indicate otherwise? Spain's link tax in 2014 led to the shutting-down of Google News Spain, and France's attempt to charge for Google News snippets led to the removal of those snippets for French sites (followed by a dip in traffic for pubs that hurt them far more than it hurt Google). What makes you think that FB/G don't have the power here?
If you don't want them to index your content and send you free traffic, you can already specify that in your robots.txt; for free. https://en.wikipedia.org/wiki/Robots_exclusion_standard
There are no ads on Google News.
There is an apparent glut of online news: supply exceeds demand and so the price has fallen.
> There are no ads on Google News.
This is the ironic part of the policy. You're right about the glut, and there was a glut in print 20 years ago, too. Google's definitely hurt businesses, but it's usually though disintermediating them (think Yelp). Google News and search don't really do that for news--snippets and headlines aren't the same as an article. I find it hard to believe that ABC's brand isn't strong enough for them to pull their content from Google and expect people to go to abc.net.au; I just don't think than can sell enough ads to give the content away.
By hurt, do you mean competed with by effectively utilizing technology to help people find information about the world from multiple sources.
There are very many news aggregators and most do serve ads next to the headlines they index. I assume that people typically link out from news aggregation sites more than into vertically-integrated services.
Perhaps the content producers / information service providers could develop additional revenue streams in order to subsidize a news aggregation public service. Micropayments (BAT, Web Monetization (ILP)), ads, paywalls, and public and private grants are sources of revenue for content producers.
I think it's disingenuous to blame news aggregation sites for the unprofitability of extremely redundant journalism. What happened to journalism? Internet. Excessive ads. Aren't we all writers these days.
Unfortunately they killed the "most cited" and was it "most in-depth" source analysis functions of Google News; and now we're stuck with regurgitated news wires and press releases and all of these eyewitness mobile phone videos with two-bit banal commentary and also punditry. How the world has changed.
So, as far as scientific experiments are concerned, it might be interesting to see what the impact of de-listing from free time sites X, Y, and Z is.
Do the papers in Australia and France now intend to compensate journal ScholarlyArticle authors whose work they summarize and hopefully at least cite the titles and URLs of, or the journals themselves?
France rules Google must pay news firms for content
Website monetization https://en.wikipedia.org/wiki/Website_monetization
Web Monetization API (ILP: Interledger Protocol)
> A JavaScript browser API which allows the creation of a payment stream from the user agent to the website.
> Web Monetization is being proposed as a #W3C standard at the Web Platform Incubator Community Group.
Interledger: Web Monetization API https://interledger.org/rfcs/0028-web-monetization/
Khan Academy, for example, accepts BAT (Basic Attention Token) micropayments/microdonations that e.g. Brave browser users can opt to share with the content producers and indexers. https://en.wikipedia.org/wiki/Brave_(web_browser)#Basic_Atte...
Web Monetization w/ Interledger should enable any payments system with low enough transaction costs ("ledger-agnostic, currency agnostic") to be used to pay/tip/donate to content producers who are producing unsensational, unbiased content that people want to pay for.
Paywalls/subscriptions and ads are two other approaches to funding quality journalism.
Should journalists pay ScholarlyArticle authors whose studies they publish summaries of without even citing the DOI/URL and Title; or the journals said ScholarlyArticles are published in? https://schema.org/ScholarlyArticle
Adafruit Thermal Camera Imager for Fever Screening
> Thermal Camera Imager for Fever Screening with USB Video Output - UTi165K. PRODUCT ID: 4579 https://www.adafruit.com/product/4579
> This video camera takes photos of temperatures! This camera is specifically tuned to work in the 30˚C~45˚C / 86˚F~113˚ F range with 0.5˚C / 1˚ F accuracy, so it's excellent for human temperature & fever detection. In fact, this thermal camera is often used by companies/airports/hotels/malls to do a first-pass fever check: If any person has a temperature of over 99˚F an alarm goes off so you can do a secondary check with an accurate handheld temperature meter.
> You may have seen thermal 'FLIR' cameras used to find air leaks in homes, but those cameras have a very wide temperature range, so they're not as accurate in the narrow range used for fever-scanning. This camera is designed specifically for that purpose!
... USB Type-C, SD Card; no price listed yet?
The end of an Era – changing every single instance of a 32-bit time_t in Linux
Thanks!
Year 2038 Problem > Solutions is already updated re: 5.6. https://en.wikipedia.org/wiki/Year_2038_problem
The article mentioned the Gates foundation which has terrible records in education initiatives:
- The big failure in small schools initiative: https://marginalrevolution.com/marginalrevolution/2010/09/th...
- The big failure in teacher evaluation initiative: https://www.businessinsider.com/bill-melinda-gates-foundatio...
- The big failure in Common Core initiative: https://www.washingtonpost.com/news/answer-sheet/wp/2016/06/...
Now it is funding anti-racist math: https://equitablemath.org/
Unbelievable.
Rather than diminishing the efforts of others, you could start helping by describing your own efforts to improve education (in order to qualify your ability to assess the mentioned and other efforts to improve education and learning)
In context to seed and series funding for a seat on a board of a for-profit venture, an NGO non-profit organization can choose whether to accept restricted donations and government organizations have elected public servant leaders who lead and find funding.
Works based on Faust: https://en.wikipedia.org/wiki/Works_based_on_Faust