Contents^
Items^
Accenture, GitHub, Microsoft and ThoughtWorks Launch the GSF
> With data centers around the world accounting for 1% of global electricity demand, and projections to consume 3-8% in the next decade, it’s imperative we address this as an industry.
> To help in that endeavor, we’re excited to announce the formation of The Green Software Foundation – a nonprofit founded by Accenture, GitHub, Microsoft and ThoughtWorks established with the Linux Foundation and the Joint Development Foundation Projects LLC to build a trusted ecosystem of people, standards, tooling and leading practices for building green software. The Green Software Foundation was born out of a mutual desire and need to collaborate across the software industry. Organizations with a shared commitment to sustainability and an interest in green software development principles are encouraged to join the foundation to help grow the field of green software engineering, contribute to standards for the industry, and work together to reduce the carbon emissions of software. The foundation aims to help the software industry contribute to the information and communications technology sector’s broader targets for reducing greenhouse gas emissions by 45% by 2030, in line with the Paris Climate Agreement.
Here's to now hand-optimized efficient EC, SHA-256, SHA-3, and Scrypt routines due to incentives. See also The Crypto Climate Accord, which is also inspired by the Paris Agreement: https://cryptoclimate.org/
... "Thermodynamics of Computation Wiki" https://news.ycombinator.com/item?id=18146854
Is 100% offset by PPAs always 200% Green?
From "Ask HN: What jobs can a software engineer take to tackle climate change?" https://news.ycombinator.com/item?id=20015801 :
> [ ] We should create some sort of a badge and structured data (JSONLD, RDFa, Microdata) for site headers and/or footers that lets consumers know that we're working toward '200% green' so that we can vote with our money.
>inspired by the Paris Agreement
So we’re just going to let China do the polluting? Maybe you didn’t know but the Paris Agreement exempted China from all standards.
No, under the Paris Agreement, countries set voluntary targets for themselves and regularly reassess.
And when China says it’s doing nothing everybody cheered.
TBF, the glut of [Chinese,] solar panels has significantly helped lower the cost of renewables; which is in everyone's interest.
They’re cheap because the secret ingredient is slavery and Uighur blood.
Rocky Linux releases its first release candidate
One of our clients pushed us toward AlmaLinux instead of CentOS 7. At the time, we vaguely suggested Rocky Linux, but noted we were kind of still waiting for the future (hence our suggestion of going with CentOS 7 and migrating later to the "true successor" of CentOS).
What is the HN community's perception of this? Is AlmaLinux a good choice? Do you believe Rocky Linux will succeed?
I’d say “neither”. CentOS would have fail had Redhat not taken over. My guess is that supporting all the companies who do not want to pay Redhat, for free, is going to push developers way pretty quickly.
Even if I’m wrong I’d still want to see at least two or three solid release before using either in production.
Just use Redhat or switch to Ubuntu. Then you can re-evaluate in five tears time.
> I’d say “neither”. CentOS would have fail had Redhat not taken over.
While it's possible that CentOS would have failed (I'm not sure slow initial releases actually indicate that), it also wasn't the only popular clone of RHEL in popular use. Scientific Linux (developed by FermiLab) was also popular and in fairly wide use. I doubt Red Hat would have made the change to CentOS that spurred all this if Scientific Linux hadn't decided not to continue and just use CentOS instead in 2019, because instead of a "crap, we have to start a new distro to provide what CentOS did" movement there would have been a much quicker and easier mass exodus to Scientific Linux.
The Scientific Linux team decided to not make a Scientific Linux 8, and instead switch over to CentOS 8[0]. Note this was announced prior to the announcement about CentOS 8 going away.
CERN et al are still deciding what to do[1].
[0]: https://listserv.fnal.gov/scripts/wa.exe?A2=SCIENTIFIC-LINUX...
[1]: https://linux.web.cern.ch/#update-on-centos-linux-strategy
USB-C is about to go from 100W to 240W, enough to power beefier laptops
What are the costs to add a USB PD module to an electronic device? https://hackaday.com/2021/04/21/easy-usb‑c-power-for-all-you...
- [ ] Create an industry standard interface for charging and using [power tool,] battery packs; and adapters
Half-Double: New hammering technique for DRAM Rowhammer bug
From "Rowhammer for qubits: is it possible?" https://amp.reddit.com/r/quantum/comments/7osud4/rowhammer_f... :
> Sometimes bits just flip due to "cosmic rays"; or, logically, also due to e.g. neutron beams and magnetic fields.
> With rowhammer, there are read/write (?) access patterns which cause predictable-enough information "leakage" to be useful for data exfiltration and privilege escalation.
> With the objective of modeling qubit interactions using quantum-mechanical properties of fields of electrons in e.g. DRAM, Is there a way to use DRAM electron "soft errors" to model quantum interactions; to build a quantum computer from what we currently see as errors in DRAM?
> If not with current DRAM, could one apply a magnetic field to DRAM in order to exploit quantum properties of electrons moving in a magnetic field?
https://en.wikipedia.org/wiki/DRAM
https://en.wikipedia.org/wiki/Row_hammer
https://en.wikipedia.org/wiki/Soft_error
https://en.wikipedia.org/wiki/Crosstalk
> [...] are there DRAM read/write patterns which cause errors due to interference which approximate quantum logic gates? Probably not, but maybe; especially with an applied magnetic field (which then isn't the DRAM sitting on our desks, it's then DRAM + a constant or variable field).
> I suppose to test this longshot theory, one would need to fuzz low-level RAM loads and search for outputs that look like quantum gate outputs. Or, monitor normal workloads which result in RAM faults which approximate quantum logic gate outputs and train a network to recognize the features.
> I am reminded of a recent approach to in-RAM computing that's not memristors.
> Soft errors caused by cosmic rays are obviously more frequent at higher altitudes (and outside of the Van Allen radiation belt).
Thought I'd ask this here as well.
Quantum tunneling was the perceived barrier at like DDR5 and higher densities FWIU? Barring new non-electron-based tech, how can we prevent adjacent electrons from just flipping at that gate grid gap size?
Other Quantum-on-Silicon approaches have coherence issues, too
Setting up a Raspberry Pi with 2 Network Interfaces as a simple router
Old but relevant, I have used an espressobin as a router for years and the performance I am getting is similar to much more expensive semi-professional routers. https://blog.tjll.net/building-my-perfect-router/
For those who wish to undergo lesser trouble to get similar performance/security there are number of Single Board Computers(SBCs) with OpenWrt support[1] sometimes with official support like those from Friendly Elec.
[1]https://openwrt.org/toh/views/toh_single-board-computers
[2]https://www.friendlyarm.com/index.php?route=product/category...
I'm constantly looking for good priced SBC with cellular connectivity - that chart shows only 2/3 with modems and all appear to be via external modules.
Anyone know of any?
> This page shows devices which have a LTE modem built in and are supported by OpenWrt.
https://openwrt.org/toh/views/toh_lte_modem_supported
It looks like this table is neither current nor complete though. And there's a different table of OpenWRT compatible devices that have a battery as well.
> [The Amarok (GL-X1200) Industrial IoT Gateway has] 2x SIM card slots for 2x 4G LTE modems (probably miniPCI-E so maybe upgradeable to 5G later), external antenna connectors for the LTE modems, MicroSD, #OpenWRT: https://store.gl-inet.com/collections/4g-smart-router/produc...
The Turris Omnia also has 4G LTE SIM card support (and LXC in their OpenWRT build). https://openwrt.org/toh/turris/turris_omnia
There's also a [Dockerized] x86 build of OpenWRT that probably also supports Mini PCI-E modules for 4G LTE, LoRa, and 5G. Route metrics determine which [gateway] route is tried first.
From "How much total throughput can your wi-fi router really provide?" https://news.ycombinator.com/item?id=26596395 :
> In 2021, most routers - even with OpenWRT and hardware-offloading - cannot actually push 1 Gigabit over wired Ethernet, though the port spec does say 1000 Mbps
What to do about GPU packages on PyPI?
“ Our current CDN “costs” are ~$1.5M/month and not getting smaller. This is generously supported by our CDN provider but is a liability for PyPI’s long-term existence.”
Wow
Bear in mind this isn't just end-users installing on their machines, it also includes continuous integration scripts that run quite frequently.
[Huge GPU] packages can be cached locally: persist ~/.cache/pip between builds with e.g. Docker, run a PyPI caching proxy,
"[Discussions on Python.org] [Packaging] Draft PEP: PyPI cost solutions: CI, mirrors, containers, and caching to scale" https://discuss.python.org/t/draft-pep-pypi-cost-solutions-c...
> Continuous Integration automated build and testing services can help reduce the costs of hosting PyPI by running local mirrors and advising clients in regards to how to efficiently re-build software hundreds or thousands of times a month without re-downloading everything from PyPI every time.
[...]
> Request from and advisory for CI Services and CI Implementors:
> Dear CI Service,
> - Please consider running local package mirrors and enabling use of local package mirrors by default for clients’ CI builds.
> - Please advise clients regarding more efficient containerized software build and test strategies.
> Running local package mirrors will save PyPI (the Python Package Index, a service maintained by PyPA, a group within the non-profit Python Software Foundation) generously donated resources. (At present (March 2020), PyPI costs ~ $800,000 USD a month to operate; even with generously donated resources).
Looks like the current figure is significantly higher than $800K/mo for science.
How to persist ~/.cache/pip between builds with e.g. Docker in order to minimize unnecessary GPU package re-downloads:
RUN --mount=type=cache,target=/root/.cache/pip
RUN --mount=type=cache,target=/home/appuser/.cache/pipWow, that RUN trick is exactly what I've been looking for! I've spent hours and hours in Docker documentation and hadn't seen that functionality.
Looks like it might be buildkit-specific?
Markdown Notes VS Code extension: Navigate notes with [[wiki-links]]
> Syntax highlighting for #tags.
What's the best way to search for #tags with VS Code? Are #tags indexed into an e.g. ctags file within a project or a directory?
> @bibtex-citations: Use pandoc-style citations in your notes (eg @author_title_year) to get syntax highlighting, autocompletion and go to definition, if you setup a global BibTeX file with your references.
Open VS Code search across files (Press Ctrl+Shift+F on windows) and type the tag that you are searching for: #tag
Simple as that https://code.visualstudio.com/docs/editor/codebasics#_search...
Ask HN: Choosing a language to learn for the heck of it
I'm a technical manager, which means I do a lot of administrative stuff and a little coding. The coding has become a nice distraction when I need to take a break.
For "real work" I write mostly Python, a lot of SQL, a little bit of Go, and some shell scripting to glue it together. I'd like to learn something I have no need of for work. If it becomes useful later, that is OK, but not a goal. The goal is in creating something just for fun. That something is undefined, so general purpose languages are the population.
I have become curious lately in Nim, Crystal, and Zig. Small, modern, high performance languages. Curiousity comes from the cases when they are mentioned here, sometime for similar reasons I list above.
Nim is on top of the list: Sort of Python like, supported on Windows (I use Win/Mac/Linux), appears to have libraries for the things I do: Process text for insights, play projects would use interesting data instead of business data.
Crystal does not support Windows (yet), but appears to closer to Ruby. Its performance may be a bit better.
Zig came on my radar recently, I know less about it, compared to the little I know of the others.
Suggestions on choosing one as a hobby language?
> Suggestions on choosing one as a hobby language?
IDK how much of a hobby it'd remain, but: Rust compiles to WASM, C++ now has auto and coroutines (and real live memory management)
"Ask HN: Is it worth it to learn C in 2020?" https://news.ycombinator.com/item?id=21878664
Show HN: Django SQL Dashboard
This looks great! In a similar vein - does anyone know of a project that will allow for a Django shell in the browser?
I know Jupyter exists - but a solution like this with the permissions would be valuable.
This launches the web-based Werkzeug debugger on Exception:
pip install django-extensions
python manage.py runserver_plus
This should run IPython Notebook with database models already imported :
python manage.py shell_plus --notebook
Interactive IPA Chart
Is there a [Linked Data] resource with the information in this interactive IPA chart (which is from Wikipedia FWICS) in addition to?:
- phoneme, ns:"US English letter combinations", []
- phoneme, ns:"schema.org/CreativeWorks which feature said phoneme", []
AFAIU, WordNet RDF doesn't have links to any IPA RDFS/OWL vocabulary/ontology yet.
Google Dataset Search
Information on how to annotate datasets: https://developers.google.com/search/docs/data-types/dataset
> We can understand structured data in Web pages about datasets, using either schema.org Dataset markup, or equivalent structures represented in W3C's Data Catalog Vocabulary (DCAT) format. We also are exploring experimental support for structured data based on W3C CSVW, and expect to evolve and adapt our approach as best practices for dataset description emerge. For more information about our approach to dataset discovery, see Making it easier to discover datasets.
For more info on those:
- W3C's Data Catalog Vocabulary: https://www.w3.org/TR/vocab-dcat-3/
- Schema.org dataset: https://schema.org/Dataset
- CSVW Namespace Vocabulary Terms: https://www.w3.org/ns/csvw
- Generating RDF from Tabular Data on the Web (examples on how to use CSVW): https://www.w3.org/TR/csv2rdf/
Use cases for such [LD: Linked Data] metadata:
1. #StructuredPremises:
> (How do I indicate that this is a https://schema.org/ScholarlyArticle predicated upon premises including this Dataset and these logical propositions?)
2. #LinkedMetaAnalyses; #LinkedResearch "#StudyGraph"
3. [CSVW (Tabular Data Model),] schema.org/Dataset(s) with per column (per-feature) physical quantity and unit URIs with e.g. QUDT and/or https://schema.org/StructuredValue metadata for maximum data reusability.
4. JupyterLab notebooks:
4a. JupyterLab Metadata Service extension: https://github.com/jupyterlab/jupyterlab-metadata-service :
> - displays linked data about the resources you are interacting with in JuyterLab.
> - enables other extensions to register as linked data providers to expose JSON LD about an entity given the entity's URL.
> - exposes linked data to the user as a Linked Data viewer in the Data Browser pane.
4b. JupyterLab Data Explorer: https://github.com/jupyterlab/jupyterlab-data-explorer :
> - Data changing on you? Use RxJS observables to represent data over time.
> - Have a new way to look at your data? Create React or lumino components to view a certain type.
> - Built-in data explorer UI to find and use available datasets.
Ask HN: Cap Table Service Recommendations
Recent founders, do you have any recommendations for services for managing a cap table? Or do you do it yourself? Any suggestions for how to choose?
Here are the "409a valuation" reviews on FounderKit: https://founderkit.com/legal/409a-valuation/reviews
Hosting SQLite databases on GitHub Pages or any static file hoster
The innovation here is getting sql.js to use http and range requests for file access rather than all being in memory.
I wonder when people using next.js will start using this for faster builds for larger static sites?
See also https://github.com/bittorrent/sqltorrent, same trick but using BitTorrent
Yeah, that was one of the inspirations for this. That one does not work in the browser though, would be a good project to do that same thing but with sqlite in wasm and integrated with WebTorrent instead of a native torrent program.
I actually did also implement a similar thing fetching data on demand from WebTorrent (and in turn helping to host the data yourself by being on the website): https://phiresky.github.io/tv-show-ratings/ That uses a protobufs split into a hashmap instead of SQLite though.
This looks pretty efficient. Some chains can be interacted with without e.g. web3.js? LevelDB indexes aren't SQLite.
Datasette is one application for views of read-only SQLite dbs with out-of-band replication. https://github.com/simonw/datasette
There are a bunch of *-to-sqlite utilities in corresponding dogsheep project.
Arrow JS for 'paged' browser client access to DuckDB might be possible and faster but without full SQLite SQL compatibility and the SQLite test suite. https://arrow.apache.org/docs/js/
> Direct Parquet & CSV querying
In-browser notebooks like Pyodide and Jyve have local filesystem access with the new "Filesystem Access API", but downloading/copying all data to the browser for every run of a browser-hosted notebook may not be necessary. https://web.dev/file-system-access/
DuckDB can directly & selectively query Parquet files over HTTP/S3 as well. See here for examples: https://github.com/duckdb/duckdb/blob/6c7c9805fdf1604039ebed...
Wasm3 compiles itself (using LLVM/Clang compiled to WASM)
Self-hosting (compilers) https://en.wikipedia.org/wiki/Self-hosting_(compilers) :
> In computer programming, self-hosting is the use of a program as part of the toolchain or operating system that produces new versions of that same program—for example, a compiler that can compile its own source code
How big of a milestone is self-hosting for a language?
Semgrep: Semantic grep for code
Is there a more complete example of how to call semgrep from pre-commit (which gets called before every git commit) in order to prevent e.g. Python print calls (print(), print \\n(), etc.) from being checked in?
https://semgrep.dev/docs/extensions/ describes how to do pre-commit.
Nvm, here's semgrep's own .pre-commit-config.yml for semgrep itself: https://github.com/returntocorp/semgrep/blob/develop/.pre-co...
I've never used the `pre-commit` framework, but it's really simple to wire up arbitrary shell scripts; check out the
`.git/hooks` directory in your repo for samples, e.g. `.git/hooks/pre-commit.sample`.
You can run any old shell script there, without having to install a python tool.
Yeah but that githook will only be installed on that one repo on that one machine. And they may have no or a different version of bash installed (on e.g. MacOS or Windows). IMHO, POSIX-compatible portable shell scripts are more trouble than portable Python scripts.
Pre-commit requires Python and pre-commit to be installed (and then it downloads every hook function).
This fetches the latest version of every hook defined in the .pre-commit-config.yml:
pre-commit autoupdate
A person could easily `ln -s repo/.hooks/hook*.sh repo/.git/hooks/` after every git clone.
Out of curiosity, Is there value in doing this over (say) running a GitHub Action post commit and failing the build if it finds something nasty?
If you can catch it before the commit is even made then why do/wait for a build?
Fair enough. Guess IDE plugins work even better for that
IDE plugins are not at all consistent from one IDE to another. Pre-commit is great for teams with different IDEs because all everyone needs to do is:
[pip,] install pre-commit
pre-commit install
# git commit
# pre-commit run --all-files
# pre-commit autoupdate
Ask HN: What to use instead of Bash / Sh for scripting?
I'm at the point where I feel a certain fatigue writing Bash scripts, but I am just not sure of what the alternative is for medium sized (say, ~150-500 LOC) scripts.
The common refrain of "use Python" hasn't really worked fantastically: I don't know what version of Python I'm going to have on the system, installing dependencies is not fun, shelling out when needed is not pleasant, and the size of program always seemingly doubles.
I'm willing to accept something that's not on the system as long as it's one smallish binary that's available in multiple architectures. Right now, I've settled on (ab)using jq, using it whenever tasks get too complex, but I'm wondering if anyone else has found a better way that should also hopefully not be completely a black box to my colleagues?
A configuration management system may have you write e.g. YAML with Jinja2 so that you don't reinvent the idempotent wheel.
It's really easy to write dangerous shell scripts ("${@}" vs ${@} for example) and also easy to write dangerous Python scripts (cmd="{}; {}").
Sarge is one way to use subprocess in Python. https://sarge.readthedocs.io/en/latest/
If you're doing installation and configuration, the most team-maintainable thing is to avoid custom code and work with a configuration management system test runner.
When you "A shell script will be fine, all I have to do is [...]" and then you realize that you need a portable POSIX shell script and to be merged it must have actual automated tests of things that are supposed to run as root - now in a fresh vm/container for testing - and manual verification of `set +xev` output isn't an automated assertion.
> avoid custom code and work with a configuration management system test runner
ansible-molecule is a test runner for Ansible playbooks that can create VMs or containers on local or remote resources.
You can definitely just call shell scripts from Ansible, but the (parallel) script output is only logged after the script returns a return code unless you pipe the script output somewhere and tail that .
> manual verification of `set +xev` output isn't an automated assertion.
From "Bash Error Handling" https://news.ycombinator.com/item?id=24745833 : you can display the line number in `set -x` output by setting $PS4:
export PS4='+(${BASH_SOURCE}:${LINENO}) '
set -x
Estonian Electronic Identity Card and Its Security Challenges [pdf]
Proud e-resident here :)
US citizen here trying to champion such a system in the US. Would you be willing to share pros and cons from your experience using the system day to day?
Are you working with a specific org or initiative? I'm also an American interested in this (and an e-resident who formerly worked for the Estonian government).
Mostly activist citizen efforts from the outside, as legislation is going to be required to appropriate funding and direction from Congress, and the legislators I interface with are busy with arguably more pressing work (unfortunate but entirely understandable, such are the times).
I intend to apply at the USDS for the Login.gov team in some capacity to help on the tech side if the necessary legislation can be put in place to support such an initiative. Their system already supports the DOD CAC (common access card), which is a short walk away from a citizen digital ID card (would be a different org and PKI root to administer and govern citizen cards, to grossly simplify).
Login.gov recently expanded to support city and local gov IAM needs (when they have ties to federal programs) [1] [2], so there is roadmap momentum and executive branch will. "Digital identity is a big deal. [3]" They're already serving 30 million users, and 500k DAUs, really just a matter of scaling up.
[1] https://www.gsa.gov/blog/2021/02/18/logingov-to-provide-auth...
[2] https://www.govloop.com/login-gov-expands-use-to-cities-stat...
I get the spirit and a m familiar with the CAC and general benefits, and using that for all .gov stuff is attractive (taxes, bills, FASFA, etc)
I think you’re glossing over two major implementation factors that need to be part of the discussion from get-go:
* how much CAC use is dependent on fairly specific govt tech infra to be widely deployed, and how will that work for everyone (ever tried to setup a CAC on a civ computer)
* key control, either extremely decentralized like the iPhone (lock yourself out of your passport?), or extremely centralized and the newest honeypot OPM holds (root cert for all e-citizen PKI). US currently doesn’t have the internal cybersec chops to run that at all (closest equivalent is CISA).
You're right to point this out, but my counter argument is that these are solvable pain points for such an implementation (either done today in competent zero trust security architectures in progressive orgs or at nation state levels such as Estonia). You won't need a CAC reader on computers, for the most part, if you have mobile apps that can perform the identity proofing (such that's already done with examples like Apple's biometrics systems, FIDO2/WebAuthn, Apply Pay, etc). You'd still have the card for interfacing at endpoints (banks, postal service, IRS/SSA offices, other trust anchors and government services endpoints). I already login to my US CBP Global Entry account with Login.gov and 2FA, why can I not today use the same IAM system to login to my Social Security account? Or my IRS tax account? Or to attest to my citizenship or other attributes that I'd normally need a certified document for (yuk!).
I'm not arguing for such a system without robust support and reasonable downgrades for failure scenarios (identity reproofing if you lose your digital ID, for example). I'm arguing for, admittedly challenging, digital ID modernization without disenfranchisement. I genuinely appreciate you pointing out the challenges, as they must be addressed.
US DHS CISA is absolutely a resource that needs to be leaned on heavily to implement what I describe, and to ensure a strong security posture throughout the federal government's infrastructure.
FWIU, DHS has funded [1] development of e.g W3C DID Decentralized Identifiers [2] and W3C Verifiable Credentials [3]:
[1] https://www.google.com/search?q=site%3Aw3.org+%22funded+by+t...
[2] https://www.w3.org/TR/did-core/
[3] https://www.w3.org/TR/vc-data-model/
Additional notes regarding credentials (certificates, badges, degrees, honorarial degrees, then-evaluated competencies) and capabilities models: https://news.ycombinator.com/item?id=19813340
westurner/blockchain-credential-resources.md: https://gist.github.com/westurner/4345987bb29fca700f52163c33...
Value storage and transmission networks have developed standards and implementations for identity, authentication, and authorization. ILP (Interledger Protocol) RFC 15 specifies "ILP addresses" for [crypto] ledger account IDs: https://interledger.org/rfcs/0015-ilp-addresses/
From "Verifiable Credentials Use Cases" https://w3c.github.io/vc-use-cases/ :
> A verifiable claim is a qualification, achievement, quality, or piece of information about an entity's background such as a name, government ID, payment provider, home address, or university degree. Such a claim describes a quality or qualities, property or properties of an entity which establish its existence and uniqueness. The use cases outlined here are provided in order to make progress toward possible future standardization and interoperability of both low- and high-stakes claims with the goals of storing, transmitting, and receiving digitally verifiable proof of attributes such as qualifications and achievements. The use cases in this document focus on concrete scenarios that the technology defined by the group should address.
FWIU, the US Department of Education is studying or already working with https://blockcerts.org/ for educational credentials.
Here are the open sources of blockchain-certificates/cert-issuer and blockchain-certificates/cert-verifier-js: https://github.com/blockchain-certificates
Might a natural-born resident get a government ID card for passing a recycling and environmental sustainability quiz.
Systemd makes life miserable, again, this time by breaking DNS
So, I made the mistake of updating my laptop from Fedora 31 to Fedora 33 last night. Normally this is fairly painless, as my laptop is one of the last machines I perform distribution upgrades. Today while doing some pole survey work out in the field, I tethered my laptop to my phone as has been done hundreds of times before. To my surprise, DNS doesn't work anymore, but only in web browsers. Both Firefox and Chrome can't resolve names anymore. Command line tools like ping and host work normally. WTF?
Why are distributions continuing to allow systemd to extend its tentacles deeper and deeper into more parts of Linux userland with poorly tested subsystem replacements for parts of Linux that have been stable for decades? Does nobody else consider this repeating pattern of rewrite-replace-introduce-new-bugs a problem? Newer is not all that better if you break what is a pretty bog standard and common use-case.
As well, Firefox now defaults to DoH (DNS over HTTPS), which may be bypassing systemd-resolved by doing DNS resolution in the app instead of calling `gethostbyname()` (`man gethostbyname`) and/or `getaddrinfo()`.
`man systemd-resolved` describes why there is new DNS functionality: security; "caching and validating DNS/DNSSEC stub resolver, as well as an LLMR and MulticastDNS resolver and responder".
From `man systemd-resolved` https://man7.org/linux/man-pages/man8/systemd-resolved.servi... :
> To improve compatibility, /etc/resolv.conf is read in order to discover configured system DNS servers, but only if it is not a symlink to /run/systemd/resolve/stub-resolv.conf, /usr/lib/systemd/resolv.conf or /run/systemd/resolve/resolv.conf
> [...] Note that the selected mode of operation for this file is detected fully automatically, depending on whether /etc/resolv.conf is a symlink to /run/systemd/resolve/resolv.conf or lists 127.0.0.53 as DNS server.
Is /etc/resolv.conf read on reload and or restart of the systemd-resolved service (`servicectl restart systemd-named`)?
Some examples of validating DNSSEC in `man delv` would be helpful.
NetworkManager (now with systemd-resolved) is one system for doing DNS configuration for zero or more transient interfaces:
man nmcli
nmcli connection help
nmcli c help
nmcli c h
nmcli c show ssid_or_nm_profile | grep -i dns
nmcli c modify help
man systemd-resolved
man delv
man dnssec-trust-anchors.dit's probably dnssec & your time is probably off.
jorunalctl -xe should be showing errors if so. probably check 'joirnalctl -xeu systemd-resolved'
use 'date' to check the time. if it's off, I use ntpdate to update my clock. but since dns isn't resolving I use 'dig pool.ntp.org @9.9.9.9' to resolve a ntp pool server ip address. then ntpdate [that up].
one of those things that hits me a bunch that I haven't automated quite yet. supposedly newer systemd can detect dnssec being bad in some cases & disable it, after a time, after a bunch of failures, but it either hasn't tripped in a number of cases for me or something else was odd in my envs. manually syncing the clock via ntp usually gets my dns working again.
> manually syncing the clock via ntp usually gets my dns working again.
Why is this necessary?
normally systemd-timedated does a good job keeping things in sync!
but some of my systems seem to have not great real-time clock batteries, and the system will forget the time. i think there might be some other circumstance that sometimes causes my system clock to be way out of whack, but i'm not sure what.
so that's why my clock doesn't work and needs sync.
the dnssec internet protocols are designed to guarantee the user that they have up to date, accurate, trustable records. this depends on your system knowing what time it is now. if your system is way ahead or way behind the actual time, the dnssec records it gets dont appear as valid. And systemd-resolved will reject them, if it is set up to respect DNSSEC.
Systemd-timedated: https://www.freedesktop.org/software/systemd/man/systemd-tim...
timedatectl set-ntp false && \
timedatectl set-ntp true
Ask HN: How bad is proof-of-work blockchain energy consumption?
I'm not a blockchain/crypto expert by any means, but I've been hearing about how much energy the proof-of-work blockchains (Bitcoin, Ethereum, NFTs) consume. Unless I'm mistaken their whole design relies on cranking through more and more CPU cycles. Should we be more concerned about this? Are the concerns overblown? Are there ways to improve it without certain crypto currencies imploding?
A rational market would be choosing an asset that offers value storage and transmission (between points in spacetime) according to criteria: "security" (security theater, infosec, cryptologic competency assessment, software assurances), "future stability" (future switching costs), and "cost".
The externalities of energy production are what must be overcome if we are to be able to withstand wasteful overconsumption of electricity. Eventually, we could all have free clean energy and no lightsabers, right?
So, we do need to minimize wasteful overconsumption. Define wasteful in terms of USD/kWHr (irregardless of industry)? In terms of behavioral economics, why are they behaving that way when there are alternatives that cost <$0.01/tx and a fairly-aggregated comprehensive n kWhr of electricity?
TIL about these guys, who are deciding to somewhat-responsibly self-regulate in the interest of long-term environmental sustainability for all of the land: "Crypto Climate Accord". https://cryptoclimate.org/
"Crypto Climate Accord Launches to Decarbonize Cryptocurrency Industry Brings together the likes of CoinShares, ConsenSys, Ripple, and the UNFCCC Climate Champions to lead sustainability in blockchain and crypto" (2021) https://bit.ly/CryptoClimateAccord
> What are the objectives of the Crypto Climate Accord? The Accord’s overall objective is to decarbonize the global crypto industry. There are three provisional objectives to be finalized in partnership with Accord supporters:
> - Enable all of the world’s blockchains to be powered by 100% renewables by the 2025 UNFCCC COP Conference
> - Develop an open-source accounting standard for measuring emissions from the cryptocurrency industry
> - Achieve net-zero emissions for the entire crypto industry, including all business operations beyond blockchains and retroactive emissions, by 2040
Similar to the Paris Agreement (2015), stakeholders appear to be setting their own targets for sustainability in accordance with the Crypto Climate Accord (2021). https://cryptoclimate.org/accord/
Someone who's not in renewables could launch e.g. a "Satoshi Nakamoto Clean Energy Fund: SNCEF" to receive donations from e.g. hash pools and connect nonprofits with sustainability managed renewables. How many SNCEFs did you give this year and why?
#CleanEnergy
Thankfully, cheap energy fit for Bitcoin mining is increasingly not found in fossil fuels, but in renewables, wasted and “stranded” sources of energy, as we’ll dive into next. [1]
[1]: https://bitcoinmagazine.com/business/bitcoin-will-save-our-e...
[2]: https://www.carbonbrief.org/solar-is-now-cheapest-electricit...
[3]: For example, to illustrate the point, I currently mine BTG on a spare GPU I had because my house needed to be heated anyways. I turn the miner on when I want heat in my house (about 600w) and turn it off when not needed... This is a micro example of what could be done across our entire society "Recent data suggest that space heating accounts for about 42 percent of energy use in U.S. residences and about 36 percent of energy use in U.S. commercial buildings."[4] . Every one of those space heaters could actually be a processor doing computation. In the future the "waster" will not be a miner but someone who turns electricity straight to heat without intermediate chains of value (such as computation or light).
[4]:https://www.epa.gov/rhc/renewable-space-heating#:~:text=Rece....
It's important to note for #3 that a mining GPU is a terribly inefficient space heater.
No, all space heaters are equally efficient. They all have perfect 100% efficiency, because they turn electrical power into heat. When your work product is heat and the waste product is also heat, then there really is no waste.
Technically in the case of cryptocurrency mining, some of the electrical power is turned into information rather than heat. In principle this reduces the amount of heat that you get, but in practice this isn’t even measurable. Most of the information is erased (discarded as useless), which turns it back into heat. Only a few hundred bits of information will be kept after successfully mining a block of transactions, and the amount of heat that costs you is fantastically small. Far smaller than you can measure.
More transistors per unit area, but also more efficient please! There should be demand for more efficient chips (semiconductors,) that are fully-utilized while depreciating on your ma's electricity bill (which is not yet (?) really determined by a market-based economy with intraday speculation to smooth over differences in supply and demand in the US). Oversupply of the electrical grid results in damage costs; which is why the price sometimes falls so low where there are intraday prices and supply has been over-subsidized pending the additional load from developing economies and EVs: Electric Vehicles.
New grid renewables (#CleanEnergy) are now less expensive than existing baseload; which makes renewables long term environment-rational and short term price-rational.
"Thermodynamics of Computation Wiki" (2018) https://news.ycombinator.com/item?id=18146854
> No, all space heaters are equally efficient. They all have perfect 100% efficiency, because they turn electrical power into heat. When your work product is heat and the waste product is also heat, then there really is no waste.
This heat must be distributed throughout the room somehow (i.e. a batteryless woodstove fan or a sterling engine that does work with the difference in entropy when there is a difference in entropy)
> Technically in the case of cryptocurrency mining, some of the electrical power is turned into information rather than heat. In principle this reduces the amount of heat that you get, but in practice this isn’t even measurable. Most of the information is erased (discarded as useless), which turns it back into heat.
See "Thermodynamics of Computation Wiki" re: a possible way to delete known observer-entangled bits while reducing heat/entropy (thus bypassing Landauer's limit for classical computation?)?
> Only a few hundred bits of information will be kept after successfully mining a block of transactions, and the amount of heat that costs you is fantastically small. Far smaller than you can measure.
Each n-symbol sequence in the hash function output does appear to have nearly equal frequency/probability of occurrence. Indeed, is Proof-of-Work worth the heat if you're not reusing the waste heat?
What does a PGP signature on a Git commit prove?
Totally though this was going to be some snarky article claiming they're worthless. Refreshing to see the article just ernestly answers the headline.
> To my knowledge, there are no effective attacks against sha1 as used by git
Perhaps im missing something, but wouldn't a chosen prefix collision be relavent here? I imagine the real reason is that cost to pull it off for sha1 is somewhere in the $10,000-$100,000 range (but getting cheaper every year) which is lots of $$$ to attack something without an obvious attack scenario that can justify it.
So the big problem with Git and SHA1 is that in many cases you are giving full control to an untrusted third party over the sha1 hash of something. For example, if you merge a binary file, it'd be quite easy for them to generate two different versions of the same file, with the same SHA1 digest, and then use the second version to cause problems in the future. You may also be able to modify a text file in the same way without getting noticed in review (I'm not up to speed on how advanced sha1 collision techniques are now).
Similarly, the git commits you merge themselves could have that done - the actual git commit serialization gives you a fair bit of ability to append stuff to it that isn't shown in the UI. That wouldn't affect the signed git commits. But it's still dubious to have the ability to change old history in a checkout.
Anyway, Git is apparently moving towards SHA256 support, so hopefully this problem will be fixed soon: https://lwn.net/Articles/823352/
> it'd be quite easy for them to generate two different versions of the same file
Citation needed. When SHA1 was cracked, it cost $110k worth of cloud computing. And there was some restriction on the two files which matched checksums. IIRC it was like the Birthday Paradox — you don’t pick one and find another sharing the same match, but you generate billions of mutations of similar binaries and statistically two would have the same checksum.
Not exactly easy, fast, cheap, or work with all use cases.
That nonce value could be ±\0 or 5,621,964,321e100; though for well-designed cryptographic hash functions it's far less likely that - at maximum difficulty - a low nonce value will result in a hash collision.
? How are nonces even involved in this?
Searching for the value to prepend or append that causes a hash collision is exactly the same as finding a nonce value at maximum difficulty (not less than the difficulty value, exactly equal to the target hash).
Mutate and check.
The term nonce has a broader meaning than how it is used in bitcoin. (Edit: i reworded this sentence from what i originally had)
That said, no, finding a collision and finding a preimage are very different things, and well the collision attacks on sha1 will involve a lot of guessing and checking, they are not generic birthday or bruteforce attacks but rely on weaknesses in sha-1 to be practical. They also do not make preimage attacks practical.
Brute forcing to find `hash(data_1+nonce) == hash(data_0)` differs very little from ``hash(data_1+nonce) < difficulty_level`. Write each and compare the cost/fitness/survival functions.
If the hash function is reversible - as may be discovered through e.g. mutation and selection - that would help find hashes that are equal and maybe also less than.
Practically, there are "rainbow tables" for very many combinations of primes and stacked transforms: it's not necessary to search the whole space for simple collisions and may not be necessary for preimages; we don't know and it's just a matter of time. "Collision attack" https://en.wikipedia.org/wiki/Collision_attack
Crytographic nonce > hashing: https://en.wikipedia.org/wiki/Cryptographic_nonce#Hashing
> Brute forcing
The attack being discussed is not a brute force attack (or not purely). If the best attack on sha1 was bruteforce than we would still be using it.
> to find `hash(data_1+nonce) == hash(data_0)` differs very little from ``hash(data_1+nonce) < difficulty_level`.
Neither of those are collision attacks (assuming you dont control the data variable). The first is a second pre-image and the second (with equality) would be a normal preimage.
The attack for sha1 under discussion (chosen prefix collision) is finding hash(a+b) == hash(c+d) where you control b and d (but not neccesarily a and c)
> Practically, there are "rainbow tables" for very many combinations of primes and stacked transforms:
What do primes or rainbow tables have to do with any of this? Primes especially. Rainbow tables are at least related to reversing hashes, if totally irrelavent to the subject at hand, but how did you get to primes?
Practically, iff browsers still relied upon SHA-1 to fingerprint and pin and verify certificates instead of the actual chain, and there were no file size limits on x.509 certificates, some fields in a cert (e.g. CommonName and SAN) would be chosen and other fields would then potentially be nonce.
In context to finding a valid cert with a known good hash fingerprint, how many prime keypairs could there be to precompute and cache/memoize when brute forcing.
"SHA-1 > Cryptanalysis and validation " does list chosen prefix collision as one of many weaknesses now identified in SHA-1: https://en.wikipedia.org/wiki/SHA-1#Cryptanalysis_and_valida...
This from 2008 re: the 200 PS3s it took to generate a rogue CA cert with a considered-valid MD5 hash: https://hackaday.com/2008/12/30/25c3-hackers-completely-brea...
... Was just discussing e.g. frankencerts the other day: https://news.ycombinator.com/item?id=26605647
Breakthrough for ‘massless’ energy storage
"Researchers from Chalmers University of Technology have produced a structural battery that performs ten times better than all previous versions"
It's 10 times more everything, just like every other battery breakthrough in the last several decades. AND, you get to use it as a building material?
- Sounds too good to be true: Check
- Sounds like every other "big" breakthrough: Check
- Article light on science and heavy on assertions: Check
- Skepticism engaged: Check
It may really be 10x, but you've mischaracterized what the paper is claiming a 10x breakthrough for. In fairness, the linked article is light on the specifics.
The claim is:
__This battery holds 10x more charge per kilo of material than previous attempts at STRUCTURAL (massless) battery materials__.
To be specific, this material holds only _ONE FIFTH_ the charge of what your smartphone's battery can hold per kilo of battery. No laws of thermodynamics is being broken and this is not at all about battery chemistry. It's all about the 'physics' of construction materials: About how this stuff is made in the factory and layered. The basic battery chemistry going on is not much different from what's been available for years - the interesting part is how this material encases it.
You can't make a car by building the chassis out of smartphone batteries. But the promise of this paper is that you CAN build the car chassis out of this battery, and even if this battery is only 20% as effective, a car chassis is rather large, and you needed it anyway, so every drop of power you can store in the chassis itself was effectively 'free' - hence the somewhat hyperbolous 'massless' terminology.
> You can't make a car by building the chassis out of smartphone batteries
They're called Structural batteries (or [micro]structural super/ultracapacitors)
"Carmakers want to ditch battery packs, use auto bodies for energy storage" (2020,) https://arstechnica.com/cars/2020/11/carmakers-want-to-ditch...
This is literally what TFA is about
OpenSSL Security Advisory
If the cloud companies just paid a team of five people 200K each to spend a year rewriting OpenSSL from scratch, they would save multiple millions in scrambling to deploy bug fixes.
Nope, they'd just create new software with different bugs that have not been discovered yet. Then we'd all be scrambling to fix those bugs.
Not if the implementation is formally verified, like miTLS [0] and EverCrypt [1]. Parts of the latter were integrated into Firefox, which even provided a performance boost (10x in one case) [2].
I think what is needed is something like EverCrypt but for TLS. Or in other words, something like miTLS but which extracts to C and/or Assembly, to avoid garbage collection and for easy interoperation with different programming languages (preferably including an OpenSSL-compatible API for backwards compatibility).
[0]: https://mitls.org/ [1]: https://hacl-star.github.io/HaclValeEverCrypt.html [2]: https://blog.mozilla.org/security/2020/07/06/performance-imp...
Formally verified against what? and what assumption were made?
miTLS is formally verified against the TLS spec on handshaking, assuming the lower-level crypto routine are good. It is not even free from timing attack.
EverCrypt have stronger proof, but it is only safe as in not crashing and correct as in match the spec in all valid input. It is not proved to be free from DoS or invalid input attack.
OpenSSL do more then TLS. Lots of interesting thing are in the cert format parsing and management.
> Formally verified against what? and what assumption were made?
Well, tell me again, what is OpenSSL formally verified against? What assumptions were made in OpenSSL?
Formal verification does not eliminate very large classes of bugs only when absolutely everything is formally verified, including timing attacks. Instead, it consistently produces more reliable software, many times even when only certain basic properties are proved (such as, memory safety, lack of integer overflows or division by zero, etc).
Formal verification can be a continuum, like testing, but proven to work for all inputs (that possibly meet certain conditions, under certain assumptions). The assumptions and properties that are proven can always be strengthened later, as seen in multiple real-world projects (such as seL4 and others).
The result is code that is almost always a lot more bug-free than code that is not formally verified. And as I said, more properties can be proven over time, especially as new classes of attacks are discovered (e.g. timing attacks, speculation attacks in CPUs, etc).
To formally verify an implementation you need a... formal description of what is to be implemented and verified.
Constructing a formal specification from RFC8446 is possible.
Constructing a formal specification for PKIX... is not. PKIX is specified by a large number of RFCs and ITU-T/ISO specs, some of which are more formal than others. E.g., constructing a formal specification for ASN.1 should be possible (though a lot of work), while constructing a formal specification for certificate validation is really hard, especially if you must support complex PKIs like DoD's. Checking CRLs and OCSP, among other things, requires support for HTTP, and even LDAP, so now you have... a bunch more RFCs to construct formal descriptions of.
And there had better be no bugs in the formal descriptions you construct from all these specs! Recall, they're mostly not formal specs at all -- they're written in English, with smatterings of ASN.1 (which is formal, though the specs for it, though they're very very good, mostly aren't formal).
The CVE in question is in the PKIX part of OpenSSL, not the TLS implementation.
What you're asking for is not 5 man-years worth of work, but tens of man-decades. The number of people with deep knowledge of all this stuff is minute as it is -- maybe just a handful, tens at most. The number of people with deep knowledge of a lot of this stuff is larger, but still minute. So you're asking to spend decades' worth of a tiny band of people's time on this project, when there are other valuable things for them to do.
The number of people who can do dev work in this space is much larger, of course -- in the thousands. But very few of them have the right expertise to work on a formal, verified implementation of PKIX.
Plus, it's all a moving target.
Sure, we could... train a lot of people just for such a project, but it takes time to do that, and it still takes time from that tiny band of people who know this stuff really well.
I'm afraid you're asking for unobtanium.
EDIT: Plus, there's probably tens of millions of current dollars' (if not more) worth of development embodied in OpenSSL as it stands. It's probably at least that much to replace it with a verified implementation, and probably much more because the value of programmers expert enough to do it is much more than the $200K/year suggested above (even if you train new ones, it would take years of training, and then they would be just as valuable). I think a proper , formally verified replacement of OpenSSL would probably run into the hundreds of millions, especially if it's one huge project since those tend to fail.
Well, sure, if your start with those premises, then I'm not surprised that you reach the conclusion that the goal is unachievable.
First of all, if constructing a formal specification for PKIX is not possible, then that should be telling you that it either needs to be simplified, better specified or scrapped altogether for something better (the latter would require an extremely large transition period, I'm imagining, so the first two are much preferred in this situation).
Otherwise, how can you be sure that any implementation in fact implements it correctly?
> And there had better be no bugs in the formal descriptions you construct from all these specs!
Well, I don't think that is true. You should in fact allow bugs in the formal description, otherwise how will you ever get anything done in such a project?
You see, having a formal description with a bug is much better than having no formal description at all.
If you have no formal description, you can't really tell if your code has bugs. If you have a buggy formal description, then you are able to catch some bugs in the implementation and the implementation can also catch some bugs in the formal description.
Also, some parts of the formal description can catch bugs in other parts of the formal description.
So the end result can be strictly better than the status quo.
> Plus, it's all a moving target.
Sure, but hopefully it's a moving target moving in the direction of simplification rather than getting more complex, otherwise things will just get worse rather than get better, regardless if we keep with the status quo or not. I'm not a TLS expert by any means but I think TLS 1.3 moved in that direction for some parts of the protocol, at least (if I'm not mistaken).
Also, I think you are not fully appreciating that formal verification can be done incrementally.
You can start by doing the minimum possible, i.e. simply verifying that your code is free of runtime errors, which would eliminate all memory safety-related bugs, including Heartbleed.
This would already be better than reimplementing in Rust, because the latter can protect from memory safety bugs but not other runtime bugs (such as division by zero, unexpected panics, etc).
BTW, this minimal verification effort would already eliminate the second bug in this security advisory.
You can then verify other simple properties, even function by function, no complicated models are even necessary at first.
For example, you could verify that your function that verifies a CA certificate, when passed the STRICT flag, is really more strict than when not passed the STRICT flag.
This would eliminate the first bug in this security advisory and all other similar bugs in the same function call-chain.
BTW, what I just said is really easy to specify, and I'm guessing it's also very easy to prove, since I'm guessing that the strict checks are just additional checks, while the normal checks are shared between the two verification modes.
Many other such properties are also easy to prove. The more difficult properties/models, or even full functional verification, can be implemented by more expert developers or even mathematicians.
I think the problem is also that OpenSSL devs, like the vast majority of devs, probably have no desire/intention/motivation/ability to do formal verification, otherwise you could even do this in the OpenSSL code base itself (although that is not ideal because verifying a program written in C requires more manual work than verifying it in a simpler language whose code extracts to C).
I'm also guessing that your budget estimate is an exaggeration since miTLS and EverCrypt, although admittedly projects whose scope has still not reached your ambitious goals (i.e. full functional verification of all layers in the stack), was probably done with a much smaller budget.
And it's not like you can't build on top of that and incrementally verify more properties over time, e.g. more layers of the stack or whatever.
You don't need a huge mega-project, just a starting point, and sufficient motivation.
> Well, sure, if your start with those premises, then I'm not surprised that you reach the conclusion that the goal is unachievable.
I didn't say unachievable. I said costly.
> First of all, if constructing a formal specification for PKIX is not possible, then that should be telling you that it either needs to be simplified, better specified or scrapped altogether for something better (the latter would require an extremely large transition period, I'm imagining, so the first two are much preferred in this situation).
The specs for the new thing would basically have to be written in Coq or similar. Even if you try real hard to keep it small and not make the... many mistakes made in PKIX's history... it would still be huge. And it would be even less accessible than PKIX already is.
> For example, you could verify that your function that verifies a CA certificate, when passed the STRICT flag, is really more strict than when not passed the STRICT flag.
That's just an argument for better testing. That's not implementation verification.
> This would eliminate the first bug in this security advisory and all other similar bugs in the same function call-chain.
Only if you thought to write that test to begin with. Writing a test for everything is... not really possible. SQLite3, one of the most tested codebases in the world, has a private testsuite that gets 100% branch coverage, and even that is not the same as testing every possible combination of branches.
> BTW, what I just said is really easy to specify, and I'm guessing it's also very easy to prove, since I'm guessing that the strict checks are just additional checks, while the normal checks are shared between the two verification modes.
It's not. The reason the strictness flag was added was that OpenSSL was historically less strict than the spec demanded. It turns out that when you're dealing with more than 30 years of history, you get kinks in the works. It wouldn't be different for whatever thing replaces PKIX.
> I think the problem is also that OpenSSL devs, like the vast majority of devs, probably have no desire/intention/motivation/ability to do formal verification, otherwise you could even do this in the OpenSSL code base itself [...]
You must be very popular at parties.
> I'm also guessing that your budget estimate is an exaggeration since miTLS and EverCrypt, [...]
Looking at miTLS, it only claims to be an implementation of TLS, not PKIX. Not surprising. EverCrypt is a cryptography library, not a PKIX library.
> That's just an argument for better testing. That's not implementation verification.
No, I'm not talking about testing, I'm talking about really basic formal verification:
forall (x: Certificate), verify_certificate(x, flags = 0) == invalid ==> verify_certificate(x, flags = STRICT) == invalid
This is trivial to specify and almost as trivial to prove to be correct for all inputs. Testing can't do that, no matter how good your testing.
> Only if you thought to write that test to begin with. Writing a test for everything is... not really possible.
I agree, but I'm not talking about testing. I'm talking about formal verification.
> It's not. The reason the strictness flag was added was that OpenSSL was historically less strict than the spec demanded. It turns out that when you're dealing with more than 30 years of history, you get kinks in the works. It wouldn't be different for whatever thing replaces PKIX.
That's totally fine and wouldn't affect the verification of that function at all. This is a very simple property to verify and it would avoid this bug and all similar bugs in that function (even if that function calls other functions, no matter how large or complex they are).
Other functions also have such easy to verify properties, this is not hard at all to come up with (although sure, some more difficult properties might be harder to verify).
> You must be very popular at parties.
I didn't want to be dismissive of OpenSSL devs or other devs in general, I just find it frustrating that there are so many myths surrounding this topic, and a lot less education, interest and investment than I think there should be, nowadays.
You are confusing verification as in "certificate" with verification as in "theorem proving", and you are still assuming a formal description of what to verify (which I'll remind you: doesn't exist). And then you go on to talk about myths and uneducated and uninterested devs. Your approach is like tilting at windmills, and will achieve exactly as much.
If I understood you correctly, then I am not confusing those two things.
Maybe you haven't noticed but I actually wrote a theorem about a hypothetical verify_certificate() function in my previous comment. Maybe you also haven't noticed, but I didn't need a formal description of how certificate validation needs to be done to write that theorem.
And I assure you, if the implementation of verify_certificate() is anything except absolute garbage, it would be very easy to prove the theorem to be correct. I actually have some experience doing this, you know? I'm not just parroting something I read, I've proved code to be 100% correct multiple times using formal verification (i.e. theorem proving) tools. That is, according to certain basic and reasonable assumptions, of course, like e.g. the hardware is not faulty or buggy while running the code, and that the compiler itself is not buggy, which would be a lot more rare than a bug in my code -- and even then, note that compilers and CPUs can be (and have been) formally verified.
Maybe you don't agree with my approach but I think it's the most realistic one for a project such as this and I am absolutely confident it would be practical, would completely eliminate most (i.e. more than 50%, at least) existing bugs with minimal verification effort (which almost anyone would be capable of doing with minimal training), and would steadily become more and more bug-free with additional, incremental verification (i.e. theorem proving) and refactoring effort.
https://project-everest.github.io/ :
> Focusing on the HTTPS ecosystem, including components such as the TLS protocol and its underlying cryptographic algorithms, Project Everest began in 2016 aiming to build and deploy formally verified implementations of several of these components in the F* proof assistant.
> […] Code from HACL*, ValeCrypt and EverCrypt is deployed in several production systems, including Mozilla Firefox, Azure Confidential Consortium Framework, the Wireguard VPN, the upcoming Zinc crypto library for the Linux kernel, the MirageOS unikernel, the ElectionGuard electronic voting SDK, and in the Tezos and Concordium blockchains.
S2n is Amazon's formally verified TLS library. https://en.wikipedia.org/wiki/S2n
IDK about a formally proven PKIX. https://www.google.com/search?q=formally+verified+pkix lists a few things.
A formally verified stack for Certificate Transparency would be a good way to secure key distribution (and revocation); where we currently depend upon a TLS library (typically OpenSSL), GPG + HKP (HTTP Key Protocol).
Fuzzing on an actual hardware - with stochastic things that persist bits between points in spacetime - is a different thing.
Funny, the first hit for that search you linked is... my comment above. The "few things" other than that are for alternatives to PKIX, which is fine and good, but PKIX will be with us for a long time yet. As for Everest, it jives with what I wrote above, that verified implementations of TLS are feasible (Everest also implements QUIC and similar), but -surprise!- not listed is PKIX.
I know, it sounds crazy, really crazy, but PKIX is much bigger than TLS. It's big. It's just big.
The crypto, you can verify. The session and presentation layers, you can verify. Heck, maybe you can verify your app. PKIX implementations of course can be verified in principle, but in fact it would require a serious amount of resources -- it would be really expensive. I hope someone does it, to be sure.
I suppose the first step would be to come up with a small profile of PKIX that's just enough for the WebPKI. Though don't be fooled, that's not really enough because people do use "mTLS" and they do use PKINIT, and they do use IPsec (mostly just for remote access) with user certificates, and DoD has special needs and they're not the only ones. But a small profile would be a start -- a formal specification for that is within the realm of the achievable in reasonably short order, though still, it's not small.
Both a gap and an opportunity; someone like an agency or a FAANG with a budget for something like this might do well to - invest in the formal methods talent pipeline and - very technically interface with e.g. Everest about PKIX as a core component in need of formal methods.
"The SSL landscape: a thorough analysis of the X.509 PKI using active and passive measurements" (2011) ... "Analysis of the HTTPS certificate ecosystem" (2013) https://scholar.google.com/scholar?oi=bibs&hl=en&cites=16545...
TIL about "Frankencerts": Using Frankencerts for Automated Adversarial Testing of Certificate Validation in SSL/TLS Implementations (2014) https://scholar.google.com/scholar?cites=3525044230307445257... :
> Our first ingredient is "frankencerts," synthetic certificates that are randomly mutated from parts of real certificates and thus include unusual combinations of extensions and constraints. Our second ingredient is differential testing: if one SSL/TLS implementation accepts a certificate while another rejects the same certificate, we use the discrepancy as an oracle for finding flaws in individual implementations.
> Differential testing with frankencerts uncovered 208 discrepancies between popular SSL/TLS implementations such as OpenSSL, NSS, CyaSSL, GnuTLS, PolarSSL, MatrixSSL, etc.
W3C ld-signatures / Linked Data Proofs, and MerkleProof2017: https://w3c-ccg.github.io/lds-merkleproof2017/
"Linked Data Cryptographic Suite Registry" https://w3c-ccg.github.io/ld-cryptosuite-registry/
ld-proofs: https://w3c-ccg.github.io/ld-proofs/
W3C DID: Decentralized Identifiers don't solve for all of PKIX (x.509)?
"W3C DID x.509" https://www.google.com/search?q=w3c+did+x509
Thanks for the link about frankencerts!
How much total throughput can your wi-fi router really provide?
Out of curiosity does some company benchmark other types of commercial networking devices like this? Things like load balancers, DDoS protection, switches, etc.? How do businesses shop for such products without having a standard measure from an independent third party?
netperf and iperf are utilities for measuring network throughput: https://en.wikipedia.org/wiki/Iperf
It's possible to approximate the https://dslreports.com/speedtest using the flent CLI or QT GUI (which calls e.g. fping and netperf) and isolate out ISP variance by running a netperf server on a decent router and/or a workstation with a sufficient NIC (at least 1Gbps). https://flent.org/tests.html
`dslreports_8dn`: https://github.com/tohojo/flent/blob/master/flent/tests/dslr...
From https://flent.org/ :
> RRUL: Create the standard graphic image used by the Bufferbloat project to show the down/upload speeds plus latency in three separate charts:
> `flent rrul -p all_scaled -l 60 -H address-of-netserver -t text-to-be-included-in-plot -o filename.png`
In 2021, most routers - even with OpenWRT and hardware-offloading - cannot actually push 1 Gigabit over wired Ethernet, though the port spec does say 1000 Mbps.
The Most Important Scarce Resource Is Legitimacy
It seems that nearly every "public good" in the Ethereum ecosystem got gobbled up by VCs and pivoted to for-profit endeavors. I don't mind VCs and I don't mind people seeking profit, but this moral high ground that Ethereum has been trying to build for itself is delusional. In practice it is just a marketing gimmick.
An astute observer may have noticed that Vitalik has a habit of coming up with definitions that only himself and the Ethereum foundation can meet the standards of.
The reality is everyone is just trying to make money. I think we are better off coming to terms with that instead of living in a collective fantasy.
Public goods are by definition non-rivalrous and non-excludable. How can they be gobbled?
https://en.wikipedia.org/wiki/Public_good_(economics)
(I'm not being pedantic here. The concept of a public good, in this technical sense, is a very important concept to have -- and it's specifically the kind of "public goods" the article was talking about, along with ideas for improving their tendency to be under-supplied. Goods that can be gobbled up by someone tend to be produced at closer to the optimum amount, since there's a less diffuse private incentive for their production.)
Public goods ... Welfare economics ... Social choice theory, Arrow's, Indifference curve: https://en.wikipedia.org/wiki/Indifference_curve
People do collectibles; commemorative plates.
A few notes on message passing
[deleted]
The big problem nobody talks about with actors and message passing is non-determinism and state.
If two otherwise independent processes A and B are sending messages to a single process C, it's non-deterministic which arrives at C first. This can create a race condition. C may respond differently depending on which messages arrives first - A's or B's. This is also effectively an example of shared mutable state - the state of C, which can be mutated whenever a message is received - is shared with A and B because they're communicating with it and their messages work differently based on the current state.
Non-determinism, race-conditions, shared mutable state: the absolute opposite of what we want when dealing with concurrency and parallelism.
> Luckily, global orders are rarely needed and are easy to impose yourself (outside distributed cases): just let all involved parties synchronize with a common process.
When there are multiple agents/actors in a distributed system, and the timestamp resolution is datetime64, and clock synchronization and network latency are variable, and non-centralized resilience is necessary to eliminate single points of failure, global ordering is impractical to impossible because there is no natural unique key with which to impose a [partial] preorder [1][2]: there are key collisions when you try and merge the streams.
Just don't cross the streams.
[1] https://en.wikipedia.org/wiki/Preorder_(disambiguation)
[2] https://en.wikipedia.org/wiki/Partially_ordered_set
The C in CAP theorem is for Consistency [3][4]. Sequential consistency is elusive because something probably has to block/lock somewhere unless you've optimally distributed the components of the CFG control flow graph.
[3] https://en.wikipedia.org/wiki/Consistency_model
[4] https://en.wikipedia.org/wiki/CAP_theorem
FWIU, TLA+ can help find such issues. [5]
[5] https://en.wikipedia.org/wiki/TLA%2B
Wouldn't a partial order be possible using logical clocks? Vector clocks probably don't satisfy "practical" but a hybrid logical clock is practical if you can assume clock synchronisation is within some modest delta.
> if you can assume clock synchronisation is within some modest delta.
From experience with clocks, you should really be actively testing the delta, and not just assuming it.
The assumption would be the limit you actively test for.
The Lamport timestamp: https://en.wikipedia.org/wiki/Lamport_timestamp :
> The Lamport timestamp algorithm is a simple logical clock algorithm used to determine the order of events in a distributed computer system. As different nodes or processes will typically not be perfectly synchronized, this algorithm is used to provide a partial ordering of events with minimal overhead, and conceptually provide a starting point for the more advanced vector clock method.
Duolingo's language notes all on one page
Succinct. What a useful reference.
An IPA (International Phonetic Alphabet) reference would be helpful, too. After taking linguistics in college, I found these Sozo videos of US english IPA consonants and vowels that simultaneously show {the ipa symbol, example words, someone visually and auditorily producing the phoneme from 2 angles, and the spectrogram of the waveform} but a few or a configurable number of [spaced] repetitions would be helpful: https://youtu.be/Sw36F_UcIn8
IDK how cartoonish or 3d of an "articulatory phonetic" model would reach the widest audience. https://en.wikipedia.org/wiki/Articulatory_phonetics
IPA chart: https://en.wikipedia.org/wiki/International_Phonetic_Alphabe...
IPA chart with audio: https://en.wikipedia.org/wiki/IPA_vowel_chart_with_audio
All of the IPA consonant chart played as a video: "International Phonetic Alphabet Consonant sounds (Pulmonic)- From Wikipedia.org" https://youtu.be/yFAITaBr6Tw
I'll have to find the link of the site where they playback youtube videos with multiple languages' subtitles highlighted side-by-side along with the video.
Found it: https://www.captionpop.com/
It looks like there are a few browser extensions for displaying multiple subtitles as well; e.g. "YouTube Dual Subtitles", "Two Captions for YouTube and Netflix"
Ask HN: The easiest programming language for teaching programming to young kids?
Hi,
I want to start a small community pilot project to help young kids, 8 and above, get interested in programming. We will use video games and robotics projects. We want to keep our tech stack as simple as possible. Here are some of the choices:
Godot + Aurdino: We can use C in Godot and Aurdino. Aurdino might be more interesting for kids as opposed neatly packaged Lego Kits.
Apple SpriteKit + Lego Mindstorm: We can use Swift with Legos. But cost will be higher.
Some of the projects we are thinking are:
Game-ish:
1. Sound visualizer like how Winamp and old school visualization were. Use speakers. And various other ideas around these concepts.
2. AR project that shows the world around you in cartoonish style. Swap faces etc.
3. Of cousre, platform games.
Robotics projects:
I see a lot of tutorials for Arduino such as robots that follow sound or light, or stuff like lights display. We will use mostly those.
Some harder project ideas I have are for drones, boats, and other navigational vehicles. This is why I want to use Arduino. But is C going to be too hard for young kids to play with?
What do you recommend? If this works, I would like to expand it and start a company around it.
awesome-python-in-education > "Python suitability for education" lists a few justifications for Python: https://github.com/quobit/awesome-python-in-education#python...
There is a Scratch Jr for Android and iOS. You can view Scratch code as JS. JS does run in a browser, until it needs WASI.
awesome-robotics-libraries: https://github.com/jslee02/awesome-robotics-libraries
FWIU, ROS (Robot Operating System) is now installable with Conda/Mamba. There's a jupyter-ros and a jupyterlab-ros extension: https://github.com/RoboStack/jupyter-ros
I just found this: https://coderdojotc.readthedocs.io/projects/python-minecraft...
> This documentation supports the CoderDojo Twin Cities’ Build worlds in Minecraft with Python code group. This group intends to teach you how to use Python, a general purpose programming language, to mod the popular game called Minecraft. It is targeted at students aged 10 to 17 who have some programming experience in another language. For example, in Scratch.
K12CS Framework has your high-level CS curriculum: https://k12cs.org/ [PDF]: https://k12cs.org/wp-content/uploads/2016/09/K%E2%80%9312-Co...
Educational technology > See also links to e.g. "Evidence-based education" and "Instructional theory" https://en.wikipedia.org/wiki/Educational_technology https://en.wikipedia.org/wiki/Educational_technology
Thank you for these resources, reviewing them now.
Yw. Np. So I just searched for "site: readthedocs.io kids python" https://www.google.com/search?q=site%3Areadthedocs.io+kids+p... and found a few new and old things:
SensorCraft (pyglet (Python + OpenGL)) from US AFRL Sensors Directorate has e.g. Gravity, Rocket Launch, and AI tutorials:
> Most people are familiar with Minecraft [...] for this project we are using a Minecraft type environment created in the Python programming language. The Air Force Research Laboratory (AFRL) Sensors Directorate located in Dayton, Ohio created this guide to inspire kids of all ages to learn to program and at the same time get an idea of what it is like to be a Scientist or Engineer for the Air Force. We created this YouTube video about SensorCraft
https://sensorcraft.readthedocs.io/en/latest/intro.html
`conda install -c conda-forge -y pyglet` should probably work. Miniforge on Win/Mac/Lin is an easy way to get Python installed on anything including ARM64 for a RPi or similar; `conda create -n scraft; conda install -c conda-forge -y python=3.8 jupyterlab jupytext jupyter-book pyglet` . If you're in a conda env, `pip install` should install things within that conda env. Here's the meta.yaml in the conda-forge pyglet-feedstock: https://github.com/conda-forge/pyglet-feedstock/blob/master/...
"BBC micro:bit MicroPython documentation" https://microbit-micropython.readthedocs.io/en/latest/
$25 for a single board-computer with a battery pack and a case (and curricula) is very reasonable: https://en.wikipedia.org/wiki/Micro_Bit
> The [micro:bit] is described as half the size of a credit card[10] and has an ARM Cortex-M0 processor, accelerometer and magnetometer sensors, Bluetooth and USB connectivity, a display consisting of 25 LEDs, two programmable buttons, and can be powered by either USB or an external battery pack.[2] The device inputs and outputs are through five ring connectors that form part of a larger 25-pin edge connector. (V2 adds a Mic and a Speaker)
Raspberry Pi for Kill Mosquitoes by Laser
This work (I think) worked with the mosquitos 30cm away with a servo scanning Pi Camera (1080p) and a 1W laser.
To work in the real world - cover a whole room or terrace - presumably a much higher resolution camera (or much faster scanning system) would be required. Even a 1W laser is dangerous to eyesight, if it was being fired at targets mingling with people.
The system could be mounted on small drones that would patrol larger areas - but the idea of robotic drones armed with lasers roaming around is beginning to sound worse than the mosquitos.
Good luck detecting a mosquito optically from a distance of several meters using a cheap camera and Raspberry Pi. Oh and you want to do from a moving drone. That will certainly make it work!
Just look at the images in the article - the guy's best result was detecting a black speck appearing on a nearby white wall with some 60-70% reliability (based on his own numbers). So you would be missing a lot of mosquitoes - but will be happy firing the laser at random shadows and what not. And that was in a completely stationary setup and controlled lab conditions, i.e. not at all something resembling a typical poorly lit room!
This article is BS. Preprints are not peer reviewed (i.e. nobody has checked anything in it - so could even be a complete hoax), it is a pretty typical gadgetry style paper (we do it because we can, not because it makes sense) you do at when you need to fill up your resume with research papers (e.g. for keeping/obtaining a job reasons).
The "save the world" (mosquito control, diseases, etc.) justification is also par for the course for this type of crappy paper. Anyone who seriously thinks that one could control mosquito problem by shooting them one by one by a laser is delusional.
But neural networks and "AI" are being used, so it has to be cutting edge groundbreaking stuff, right?
BTW, this nonsense idea has been floated as a publicity stunt a few years ago (including a slow motion video of a laser burning off wing of a mosquito in flight) and it seems that some Russian PhD student from a fairly obscure uni either didn't do their research or has reinvented the wheel (or just plain copied the thing without attribution). The list of irrelevant or only very tangentially relevant (it is about mosquitoes, so in scope, right?) references is a dead giveaway there (paper on mosquitoes spreading zika? seriously?).
Here, it was even on National Geographic in 2010(!): https://www.youtube.com/watch?v=BKm8FolQ7jw
Oh and that was supposed to be a handheld device to boot. With the same "save the world from malaria" spiel too. I wonder what are the owners of the company that was pushing this concept to investors back then trying to sell today ...
There are actually multiple videos on Youtube showing products from different companies that were attempting to push this as some sort of viable concept.
> using a cheap camera
even if you had a RED Komodo feeding uncompressed 4K DCI 60fps video to a pci-express bus capture card, the sensor resolution and tiny size of mosquitoes means that unless the lighting conditions are just right, and the mosquito is somehow highlighted against a background, it's going to be very hard to pick them out at distances of 2 or more meters.
and that's before you get into the software problem of processing the fire hose of data that is 4096x2160 at 60fps raw. and the hardware cost of a very serious workstation class PC capable of taking the capture at 1:1 realtime.
possibly a lidar based sensor or something might be more suitable to locating the x/y/z position of mosquitoes in a few meter area.
Mosquitos, as we all know, have a highly distinctive auditory signature.
A phased microphone array is the only sensible approach to localized mosquito detection. It would probably work reasonably well.
The problem is that the entire field is patent encumbered, because Myhrvold's company Intellectual Ventures has done some research on it, and no on in their right mind would go up against those guys.
Yeah, they already did sharks with lasers. IDK what the licensing terms are on that
ONE MILLION DOLLARS
Donate Unrestricted
The article mentioned the Gates foundation which has terrible records in education initiatives:
- The big failure in small schools initiative: https://marginalrevolution.com/marginalrevolution/2010/09/th...
- The big failure in teacher evaluation initiative: https://www.businessinsider.com/bill-melinda-gates-foundatio...
- The big failure in Common Core initiative: https://www.washingtonpost.com/news/answer-sheet/wp/2016/06/...
Now it is funding anti-racist math: https://equitablemath.org/
Unbelievable.
Rather than diminishing the efforts of others, you could start helping by describing your own efforts to improve education (in order to qualify your ability to assess the mentioned and other efforts to improve education and learning)
In context to seed and series funding for a seat on a board of a for-profit venture, an NGO non-profit organization can choose whether to accept restricted donations and government organizations have elected public servant leaders who lead and find funding.
Works based on Faust: https://en.wikipedia.org/wiki/Works_based_on_Faust
Bitcoin Is Time
I'm ready to be downvoted to oblivion, especially by all the Bitcoin purists, but I hope my message sparks at least some curiosity.
Bitcoin is getting "weaker" every year for several reasons:
- emerging centralization due to economies of scale. If this trend continues the mining power will be so consolidated that a 51% attack will be likely. The Nakamoto coefficient is already at 4, and tending towards 3.
- energy usage due to Proof of Work is growing astronomically, and the higher the price of bitcoin, the less incentive there is to use renewable energy. Bitcoin uses more energy than the country of Argentina.
- transaction times are SLOW, and expensive. The lightning network is incredibly buggy and won't actually solve the problems it's promising.
There is a better alternative. RaiBlocks, named nano since 2018. It got a bad rap due to the BitGrail hack, but it's picking up steam again. The developer community is great, the main dev team on the Nano protocol has been consistently chugging along regardless of the 3 years of crypto winter on a shoestring budget and the nano community is made up of users, not speculators.
This article sums it up quite nicely: https://senatusspqr.medium.com/why-nano-is-the-ultimate-stor...
I'm happy to receive downvotes, but all I ask in return is that you give it a read with an open mind.
> The lightning network is incredibly buggy and won't actually solve the problems it's promising.
Not an expert but I found that a readable summary of different layer 2 scaling strategies and why Ethereum developers prefer Rollups instead of Channels (like Lightning):
"Bitcoin scalability problem" could link to the Ethereum design docs: https://en.wikipedia.org/wiki/Bitcoin_scalability_problem
The Ethereum design docs could link to direct-listed premined [stable] coins as a solution for Proof of Work and TPS reports: https://github.com/flare-eng/coston#smart-contracts-with-xrp
(edit) re: n-layer solutions: The https://interledger.org/ RFCs and something like Transaction Permission Layer (TPL) will probably be helpful for interchain compliance.
> Interledger is not tied to a single company, blockchain, or currency.
From https://tplprotocol.org/ :
> The challenge: Current blockchain-based protocols lack an effective governance mechanism that ensures token transfers comply with requirements set by the project that issued the token.
> Projects need to set requirements for a variety of reasons. For instance, remaining compliant with securities laws, limiting transfer to beta testers, or limiting transfer to a particular geo-spatial location. Whatever your reason, if a requirement can be verified by a third-party, TPL will be able to help.
In the US, S-Corps can't have international or more than n shareholders, for example; so if firms even wanted to issue securities on a first-layer network, they'd need an extra-chain compliance mechanism to ensure that their issuance is legal pursuant to local, sovereign, necessary policies. Re-issuing stock certificates is something that has to be done sometimes. When is it possible to cancel outstanding tokens?
Foundational Distributed Systems Papers
From "Ask HN: Learning about distributed systems?" https://news.ycombinator.com/item?id=23932271 :
> Papers-we-love > Distributed Systems: https://github.com/papers-we-love/papers-we-love/tree/master...
> awesome-distributed-systems also has many links to theory: https://github.com/theanalyst/awesome-distributed-systems
And links to more lists of distributed systems papers under "Meta Lists": https://github.com/theanalyst/awesome-distributed-systems#me...
In reviewing this awesome list, today I learned about this playlist: "MIT 6.824 Distributed Systems (Spring 2020)" https://youtube.com/playlist?list=PLrw6a1wE39_tb2fErI4-WkMbs...
> awesome-bigdata lists a number of tools: https://github.com/onurakpolat/awesome-bigdata
Low-Cost Multi-touch Whiteboard using the Wiimote (2007) [video]
"Interactive whiteboard" / "smart board" https://en.wikipedia.org/wiki/Interactive_whiteboard
Wii Remote > Features > Sensing: https://en.wikipedia.org/wiki/Wii_Remote#Sensing
.. > Third-Party Development describes a number of applications for IR/optical tracking with an array of nonstationary emitters: https://en.wikipedia.org/wiki/Wii_Remote#Third-party_develop...
Augmented Reality (AR) > Technology > Tracking: https://en.wikipedia.org/wiki/Augmented_reality#Tracking
... links to "VR positional tracking" which does have headings for "Optical" and "Sensor fusion": https://en.wikipedia.org/wiki/VR_positional_tracking
How to Efficiently Choose the Right Database for Your Applications
I kinda disagree with separate branch for "document database" for Mongo. Mongo is a key-value storage, with a thin wrapper that converts BSON<->JSON, and indices on subfields.
You can achieve exactly the same thing with PostgreSQL tables with two columns (key JSONB PRIMARY KEY, value JSONB), including indices on subfields. With way more other functionality and support options.
> You can achieve exactly the same thing with PostgreSQL tables with two columns (key JSONB PRIMARY KEY, value JSONB), including indices on subfields. With way more other functionality and support options.
PostgreSQL docs > "JSON Functions and Operators" https://www.postgresql.org/docs/current/functions-json.html
MongoDB can do jsonSchema:
> Document Validator¶ You can use $jsonSchema in a document validator to enforce the specified schema on insert and update operations:
db.createCollection( <collection>, { validator: { $jsonSchema: <schema> } } )
db.runCommand( { collMod: <collection>, validator:{ $jsonSchema: <schema> } } )
Looks like there are at least 2 ways to handle JSONschema with Postgres: https://stackoverflow.com/questions/22228525/json-schema-val... ; neither of which are written in e.g. Rust or Go.
Is there a good way to handle JSON-LD (JSON Linked Data) with Postgres yet?
There are probably 10 comparisons of triple stores with rule inference slash reasoning on data ingress and/or egress.
A Data Pipeline Is a Materialized View
Sometime when I am old(er) and (somehow?) have more time, I'd like to jot down a "Rosetta Stone" of which buzzwords map to the same concepts. So often we change our vocabulary every decade without changing what we're really talking about.
Things started out in a scholarly vein, but the rush of commerce hasn't allowed much time to think where we're going. — James Thornton, Considerations in computer design (1963)
Like a Linked Data thesaurus with typed, reified edges between nodes/concepts/class_instances?
Here's the WordNet RDF Linked Data for "jargon"; like the "Jargon File": http://wordnet-rdf.princeton.edu/lemma/jargon
A Semantic MediaWiki Thesaurus? https://en.wikipedia.org/wiki/Semantic_MediaWiki :
> Semantic MediaWiki (SMW) is an extension to MediaWiki that allows for annotating semantic data within wiki pages, thus turning a wiki that incorporates the extension into a semantic wiki. Data that has been encoded can be used in semantic searches, used for aggregation of pages, displayed in formats like maps, calendars and graphs, and exported to the outside world via formats like RDF and CSV.
Google Books NGram viewer has "word phrase" term occurrence data by year, from books: https://books.google.com/ngrams
There’s no such thing as “a startup within a big company”
When I was at PowerBI in Microsoft, all the execs hailed it as Startup within Microsoft. Come work here instead of Uber. I worked like a dog, sometimes till 2am in morning. My manager would routinely ask us to come on weekends. I was naive, I thought we are growing customer base, this is what a startup looks like.
The ultimate realization was in a startup you have equity, a decent amount in a good startup. At Microsoft it was a base salary and set amount of stock. What we did moved very little of the top revenue metric. It made little difference if I worked like a dog, or slacked. The promos were very much “buddy buddy” system.
In the end I realized you can’t have startups in big companies (esp as an engineer you don’t have the huge upside if the startup is successful, your upside is capped)
Startups work because you have skin in the game, when you build something people want, you get to reap rewards proportional to it. That correlation and feedback loop is very important.
At big companies you don’t have the the same correlation. Some big shot exec they hired reaps far far more on the work you did.
Equity is what builds wealth.
>The ultimate realization was in a startup you have equity, a decent amount in a good startup. At Microsoft it was a base salary and set amount of stock.
I joined a startup in 1999. There were 3 founders and I was employee #2 after that. I received a ton of options (this was before RSUs became popular). We had a great product and a great team, but 18 months later ran out of money and unfortunately it was right after the dotcom implosion of early 2001 when funding had completely dried up.
My takeaway was that base salary is actually the most important component of TC. Cash bonus based on some metric that you control comes second. Equity comes third. If you work for a FAANG, maybe equity can move higher up (though it remains to be seen how long this will be true).
Outside of FAANG (and top executives at F500 sized public companies) very few people are getting rich off of the "equity" component of their TC. The vast majority of startups go bust before IPO or acquisition.
Time is the most valuable commodity you have, don't squander it for lottery tickets and empty promises.
>Time is the most valuable commodity you have, don't squander it for lottery tickets and empty promises.
That goes the other way around too, the only way to have massive amounts of free time without going FIRE is to win the lottery. So why not throw the dice once or twice when you're young and then settle into a stable corporate job if it doesn't pay out?
Modern Silicon Valley lets you do both. Why not throw the dice and still make around $200k cash?
As a [good] engineer you can have your cake and eat it too. Sure it won’t make you a billionaire but you can live a comfy life, maybe win the lottery, and retire at 50 if the lottery fails.
Living and working elsewhere with the wages of the region reduces expenses and opportunities; but the wealth of educational resources online [1][2] does make it feasible to even bootstrap a company on the side. Do you need to borrow money to scale quickly enough to pay expenses with sufficient cash flow for the foreseeable future?
Income sources: Passive income, Content, Equity that's potentially worth nothing, a backtested diversified portfolio (Golden Butterfly or All Weather Portfolio and why?) of sustainable investments, Business models [3]; Software implementations of solutions to businesses, organizations, and/or consumers' opportunities
Single-payer / Universal Healthcare is a looming family expense for many entrepreneurs; many of whom do get into entrepreneurship later in life.
Small businesses make up a significant portion of GDP. Small businesses have to have to accept risk.
There's still opportunity in the world.
[1] Startup School > Curriculum https://www.startupschool.org/curriculum
[2] https://www.ycombinator.com/library
[3] "Business models based on the compiled list at [HN]" https://gist.github.com/ndarville/4295324
From "Why companies lose their best innovators (2019)" https://news.ycombinator.com/item?id=23887903 :
> "Intrapreneurial." What does that even mean? The employee, within their specialized department, spends resources (time, money, equipment) on something that their superior managers have not allocated funding for because they want: (a) recognition; (b) job security; (c) to save resources such as time and money; (d) to work on something else instead of this wasteful process; (e) more money.
I’d you are going to start something on the side, carefully read your current employment agreement, particularly the part about IP assignment. Most Silicon Valley companies I’ve worked with, including FAANGs, claim ownership of everything you produce, inside or outside of employment, at home or in office, using their equipment or yours. You don’t want to lick into a unicorn idea and have your former employer’s lawyers send you that letter...
Unusual data point (I'm not sure if South African practice is applicable elsewhere) , but most bigger corps I've worked at have provided a mechanism to declare external interests, pending management approval, to work around these global clauses.
On paper any IP would be default belong to the Corp, but after some discussion/negotiations with your line manager and his higher-ups, you could do the paper work to declare.
Are these actually enforceable?
It probably doesn’t matter in practice. They will have more lawyers and more money to burn on legal process than you. Who will go bankrupt first fighting a legal battle between you and, say, Apple?
In practice, I don't think I've ever seen a SV example of that kind of litigation from garage development of IP. Maybe I've just missed them?
Theft of IP from one company to a new company on the other hand there are multiple public examples of.
The only time they'd actually sue is if you made something worth money. in that case, there's a good chance you can get investors to pay the lawsuit.
It does matter in practice in California.
If its "related" to your employers business yes - unless you agree a variation of the contract.
Ask HN: Keyrings: per-package/repo; commit, merge, and release keyrings?
Are there existing specs for specifying per-package release keyrings and per-repo commit and merge keyrings?
Keyring: a collection of keys imported into a datastore with review.
DevOpsSec; Software Supply Chain Security
Packages {X, Y, Z} in Indexes {A, B, C} are artifacts that are output from Builds (on workstations or servers with security policies) which build a build script (which is often deliberately not specified with a complete programming language in order to minimize build complexity; instead preferring YAML) which should be drawn from a stable commit hash in a Repository (which may be a copy of technically zero or more branches of a Repository hosted centrally next to Issues and Build logs and Build artifact Signing Keys).
Maxmimally, are there potentially more keyrings (or key authorization mappings between key and permission) than (1) commit; (2) merge; and (3) release?
Source Projects: Commit, Merge, [Run Build, Login to post-build env], Release (and Sign) package
Downstream Distros: Commit, Merge, [Run Build, Login to post-build env], Release (and Sign) package for the {testing, stable, security} (Signed) Index catalogs
Threat Actors Now Target Docker via Container Escape Features
It may be quipped that Docker or Kubernetes is remote code execution (RCE) as a service!
The vulnerability pointed out here is server operators who have exposed Docker's API to the public internet, allowing anyone to run a container. The use of a privileged container is just icing on the cake, and probably not nessecary for a cryptominer.
A more subtle and interesting attack vector is genuine container imags that become compromised, much like any other package manager, particularly ones that search for the Docker API on the host's internal IP address, hoping only public network access is firewalled.
Docker engine docs > "Protect the Docker daemon socket" https://docs.docker.com/engine/security/protect-access/
dev-sec/cis-docker-benchmark /controls: https://github.com/dev-sec/cis-docker-benchmark/tree/master/...
Eh. This advice is less practical than it’s made to seem. Like it “works” but it’s not really usable for anything other than connecting two privileged apps over a hostile network.
* Docker doesn’t support CRLs so any compromised cert means reissuing everyone’s cert.
* Docker’s permissions are all or nothing without a plug-in. And if you’re going that route the plug-in probably has better authentication.
* Docker’s check is just “is the cert signed by the CA” so you have to do one CA per machine / group homogeneous machines.
* You either get access to the socket or not with no concept of users so you get zero auditing.
* Using SSH as transport helps but then you have to also lock down SSH which isn’t impossible but more work and surface area to cover than feels necessary. Also since your access is still via the Unix socket it’s all or non permissions again.*
django-ca is one way to manage a PKI including ACMEv2, OCSP, and a CRL (Certificate Revocation) list: https://github.com/mathiasertl/django-ca
"How can I verify client certificates against a CRL in Golang?" mentions a bit about crypto/tls and one position on CRLs: https://stackoverflow.com/questions/37058322/how-can-i-verif...
CT (Certificate Transparency) is another approach to validating certs wherein x.509 cert logs are written to a consistent, available blockchain (or in e.g. google/trillian, a centralized db where one party has root and backup responsibilities also with Merkle hashes for verifying data integrity). https://certificate.transparency.dev/ https://github.com/google/trillian
Does docker ever make the docker socket available over the network, over an un-firewalled port by default? Docker Swarm is one config where the docker socket is configured to be available over TLS.
Docker Swarm docs > "Manage swarm security with public key infrastructure (PKI)" https://docs.docker.com/engine/swarm/how-swarm-mode-works/pk... :
> Run `docker swarm ca --rotate` to generate a new CA certificate and key. If you prefer, you can pass the --ca-cert and --external-ca flags to specify the root certificate and to use a root CA external to the swarm. Alternately, you can pass the --ca-cert and --ca-key flags to specify the exact certificate and key you would like the swarm to use.
Docker ("moby") and podman v3 socket security could be improved:
> From "ENH,SEC: Create additional sockets with limited permissions" https://github.com/moby/moby/issues/38879 ::
> > An example use case: securing the Traefik docker driver:
> > - "Docker integration: Exposing Docker socket to Traefik container is a serious security risk" https://github.com/traefik/traefik/issues/4174#issuecomment-...
> > > It seems it only require (read) operations : ServerVersion, ContainerList, ContainerInspect, ServiceList, NetworkList, TaskList & Events.
> > - https://github.com/liquidat/ansible-role-traefik
> > > This role does exactly that: it launches two containers, a traefik one and another to securely provide limited access to the docker socket. It also provides the necessary configuration.
> > - ["What could docker do to make it easier to do this correctly?"] https://github.com/Tecnativa/docker-socket-proxy/issues/13
> > - [docker-socket-proxy] Creates a HAproxy container that proxies limited access to the [docker] socket
This looks like the attacker is just using a publicly exposed Docker API honeypot to run a new Docker container with `privileged: true`. I don't see why that's particularly interesting given that they could just bind-mount the host / filesystem with `pid: host` and do pretty much whatever they want?
It would be much more interesting to see a remote-code-execution attack against some vulnerable service deploying a container escape payload to escalate privileges from the Docker container to the host.
Exposed on the WAN is obviously bad, but how do you keep your own containers from calling those APIs? Yes, you don't mount the docker socket in the container, but what about the orchestrator APIs?
With kubernetes, you enable RBAC, and only allow each pod the absolute minimum of access required.
In my own setup, I have only five pods with any privileges: kubernetes-dashboard, nginx-ingress, prometheus, cert-manager, and the gitlab-runner, and of those only kubernetes-dashboard can modify existing pods in all spaces, cert-manager can only write secrets of its own custom type, and gitlab-runner can spawn non-privileged pods in its own namespace (but not mount anything from the host, nor any resources from any other namespace).
And when building docker images in the CI, I use google’s kaniko to build docker images from within docker without any privileges (it unpacks docker images for building and runs them inside the existing container, basically just chroot).
All these APIs are marked very clearly and obviously as DO NOT MAKE AVAILABLE PUBLICLY. If you still make them public, well it’s pretty much your own fault.
Does RBAC not limit these by default? Does cert-manger not already give itself restricted permission on install? Do I need to fix up my cluster right now? If so, do you have any example RBAC yamls? :D
> And when building docker images in the CI, I use google’s kaniko to build docker images from within docker without any privileges (it unpacks docker images for building and runs them inside the existing container, basically just chroot).
You can also use standalone buildkit which comes with the added benefit of being able to use the same builder locally natively.
No RBAC doesn't automatically do this. And many publicly available Helm charts are missing these basic security configurations. You should use Gatekeeper or similar to enforce these settings throughout your cluster.
Gatekeeper docs: https://open-policy-agent.github.io/gatekeeper/website/docs/
Gatekeeper src: https://github.com/open-policy-agent/gatekeeper
awesome-container-security: https://github.com/kai5263499/awesome-container-security
"container-security" GitHub label: https://github.com/topics/container-security
In somewhat related news, rootless Docker support is no longer an experimental feature. Has anyone used it and liked/disliked it?
I have, and it's great/you forget about it pretty quickly. I've actually gone (rootful?)docker->(rootless)podman->(rootless)docker, and I've written about it:
- https://vadosware.io/post/rootless-containers-in-2020-on-arc...
- https://vadosware.io/post/bits-and-bobs-with-podman/
- https://vadosware.io/post/back-to-docker-after-issues-with-p...
I still somewhat long after podman's lack of a daemon but some of the other issues with it currently leave it completely out for me (now that docker has caught up) -- for example lack of linking support.
Podman works great outside of those issues though, you just have to be careful if it does something in a way that expects docker semantics (usually the daemon or an unix socket on disk somewhere). You'll also find some cutting edge tools like dive[0] also have some support for podman[1] but it's not necessarily the usual case.
podman v3 has a docker-compose compatible socket. From https://news.ycombinator.com/item?id=26107022 :
> "Using Podman and Docker Compose" https://podman.io/blogs/2021/01/11/podman-compose.html
Ask HN: What security is in place for bank-to-bank EFT?
When I set up an ETF on a bank's website, all I need to enter is the other bank's routing number and account number, which can be readily found on a paper check. Then you can transfer money from one bank to another... What security and authentication is in place to prevent fraud? In case of fraud, is the victim guaranteed to get the money back?
AFAIU, no existing banking transaction systems require the receiver to confirm in order to receive a funds transfer.
You can create a "multisig" DLT smart contract that requires multiple parties' signatures before the [optionally escrowed] funds are actually transferred.
EFT: Electronic Funds Transfer: https://en.wikipedia.org/wiki/Electronic_funds_transfer
As far as permissions to write to the account ledger: Check signatures are scanned. Cryptoasset keys are very long, high-entropy "passwords". US debit cards are chip+pin; it's not enough to just copy down the card number (and CVV code).
Though credit cards typically are covered by fraud protection, debit card transactions typically aren't: hopefully something will be recovered, but AFAIU debit txs might as well be as unreversible as cryptoasset transactions.
TPL: Transaction Permission Layer is one proposed system for permissions in blockchain; so that e.g. {proof of residence, receiver confirmation, accredited investor status, etc.} can be necessary for a transaction to go through.
ILP: Interledger Protocol > RFC 32 > "Peering, Clearing and Settling" describes how ~EFT with Interledger works: https://interledger.org/rfcs/0032-peering-clearing-settlemen...
Podman: A Daemonless Container Engine
Is the title of this page out of date?
AFAIU, Podman v3 has a docker-compose compatible socket and there's a daemon; so "Daemonless Container Engine" is no longer accurate.
"Using Podman and Docker Compose" https://podman.io/blogs/2021/01/11/podman-compose.html
Podman does not run in a daemon-mode by default. There's a dbus-activated service unit that activates the daemon when you try to use Docker Compose with it.
It is almost 100% compatible with docker. I only had to do minimal changes in my Dockerfiles to use it.
But not Buildkit and docker-compose.
Podman v3 is compatible with docker-compose (but not yet swarm mode, FWIU), has a socket and a daemon that services it.
Buildah (`podman buildx`, `buildah bud --arch arm64`) just gained multiarch build support; so also building arm64 containers from the same Dockerfile is easy now. https://github.com/containers/buildah/issues/1590
IDK what BuildKit features should be added to Buildah, too?
> IDK what BuildKit features should be added to Buildah, too?
Cache mounts are what we rely on for incremental rebuilds.
Cambridge Bitcoin Electricity Consumption Index
Ok, I have to admit that I am one of those people who knew about BTC very, very early but never bought any. Now, of course I should have mined at least 100 back in the days, but anyways.
The reason I have never mined nor bought into it is that I still, to this day struggle to come up with a real reason to do so.
Am I getting too old? Do I not see the "huge potential" what it may become? Don't I want to get freaking rich? I simply can't answer what BTCs and other crypto coins are actually good for. Can someone help me out here?
I'm in your boat, when it was first explained to me in a hacker space at college when it was first growing and priced at a few cents per 100++ Bitcoin. All I could see was a waste of energy and resources (time, energy, hardware).
It will always be a drain on society, and it scares me that it hasn't failed because we see it draining our GPU industry to the point that market prices have gone insane and availability is practically non-existent / all left up to chance.
It does take a rocket science to see the trends, as long as it's tied to needing hardware to compute arbitrary calculations to work... It's going to waste computer resources and energy. Both which are finite resources... Yes finite, solar cells aren't free to make, they don't last forever after you make them. Same with any other green technology. Maybe if fusion was our only power source it would be less of a concern, but we still haven't cracked Q=1, so... Yes it is a waste of finite resources.
It’s not draining the GPU industry. No one buys GPUs for mining Bitcoin anymore, they use ASICs.
Even as just ASICs there's something of a drain, Bitcoin ASICs consume fab production which means less remaining production capacity for GPUs.
This is an equilibrium that should eventually fix itself, but while Bitcoin and crypto mining is growing they are making a material dent in the global semiconductor economy.
Cryptoasset mining creates demand for custom chip fab (how different are mining rigs from SSL/TLS accelerator expansion cards), which is definitely not zero sum: more revenue = more opportunities.
https://en.wikipedia.org/wiki/Price_elasticity_of_supply
With insufficient demand, a market does not develop into a sustainable market. "Rule of three (economics)" says that markets are stable with 3 major competitors and many smaller competitors; nonlinearity and game theory.
https://en.wikipedia.org/wiki/Rule_of_three_(economics)
We've always had custom chip fab, but the prices used to be much higher. Proof of Work (and Proof of Research) incentivize microchip and software energy efficiency; whereas we had observed and been most concerned with doublings in transistor density.
FWIU, it's now more sustainable and profitable to mine rare earth elements from recycled electronics than actually digging real value out of the earth?
Compared to creating real value by digging for gold, how do we value financial services?
Bitcoin's fundamental value is negative given its environmental impact
If the price of energy is calculated with corresponding carbon tax included, shouldn't Bitcoin be neutral?
> If the price of energy is calculated with corresponding carbon tax included, shouldn't Bitcoin be neutral?
Yes, but there are vastly more energy efficient substitute DLTs with near-zero switching costs. Litecoin and scrypt (instead of AES256), for example.
Apply a USD/kWhr threshold across all industries.
Is this change (and focus on the external costs of energy production) more the result of penalties or incentives?
Pre-mined coins are vastly more energy efficient (with tx costs <1¢ and similarly minimal kWhr/tx costs), but the market doesn't trust undefined escrow terms that are fair game in commodities and retail markets.
We have trouble otherwise storing energy from noon to commute and dinner time; whereas a commodity like grain may keep for quite awhile.
Bitcoin serves as a demand subsidy when heavily-subsidized energy prices crash due to oversupply (that we should recognize as temporary because we are moving to electric vehicles and we need to reach production volumes so that, in comparison to alternatives, renewables are now more cost effective)
In the US, we have neither carbon taxes nor intraday prices. The EU has carbon taxes and electrical energy markets.
>In the US, we have neither carbon taxes nor intraday prices.
Like, no intraday electricity market?
Ask HN: What are some books where the reader learns by building projects?
2021 Edition. This is a continuation of the previous two threads which can be found here:
https://news.ycombinator.com/item?id=22299180
https://news.ycombinator.com/item?id=13660086
Other resources:
https://github.com/danistefanovic/build-your-own-x
https://github.com/AlgoryL/Projects-from-Scratch
https://github.com/tuvtran/project-based-learning
"Agile Web Development with Rails [6]" (2020) teaches TDD and agile in conjunction with a DRY, CoC, RAD web application framework: https://g.co/kgs/GNqnWV
Is it wrong to demand features in open-source projects?
Yes, it's wrong to demand something for nothing: that's entitlement, not business (which involves some sort of equitable exchange of goods and/or services).
Better questions: How do I file a BUG report issue, create a feature ENHancement request issue, send a pull request with a typo fix, write DOCs and send a PR, write test cases for a bug report?
How can I sponsor development of a feature?
A project may define a `.github/FUNDING.yml`, which GitHub will display on the 'Sponsor' tab of the GitHub project. A project may also or instead include funding information in their /README.md.
How do I ask IRC or a mailing list or issues how much and how long it would cost to develop a feature, if somebody had some international stablecoin and a limited term agreement?
The answer may be something like, "thanks for the detailed use case or user story, that's on our roadmap, there are major issues blocking similar features and that's where the expense would be."
CompilerGym: A toolkit for reinforcement learning for compiler optimization
What a great idea. Would per-instruction/opcode costs make this easier to optimize?
Are MuZero or the OpenAI baselines useful for RL compiler optimization?
Turning desalination waste into a useful resource
Tangentially related: there are millions of abandoned oil wells emitting greenhouse gases like methane across the US. Vice had a great video about this last month, but it didn't get much attention here on HN. [1] [2]
Evcxr: A Rust REPL and Jupyter Kernel
I have been using evcxr for the last 2-3 weeks and I'm loving it. There are two things that bother me: (a) fancy UI components seem to only work with Python and C++ kernel (not theoretically, but with available packages), (b) while you can redefine values and functions when you're iterating, you cannot do so with structs (yes, big structs go to a local package that I depend on, but sometimes I have ad-hoc utility structs).
Here's the xeus-cling (Jupyter C++ Kernel) source: https://github.com/jupyter-xeus/xeus-cling/tree/master/src
Do any of the other non-Python Jupyter kernels have examples of working fancy UI components? https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
Jupyter kernels implement the Jupyter kernel message spec. Introspection, Completion: https://jupyter-client.readthedocs.io/en/latest/messaging.ht...
Debugging (w/ DAP: Debug Adapter Protocol) https://jupyter-client.readthedocs.io/en/latest/messaging.ht...
A `display_data` Jupyter kernel message includes a `data` key with a dict value: "The data dict contains key/value pairs, where the keys are MIME types and the values are the raw data of the representation in that format." https://jupyter-client.readthedocs.io/en/latest/messaging.ht...
This looks like it does something with MIME bundles: https://github.com/jupyter-xeus/xeus-cling/blob/00b1fa69d17b...
ipython.display: https://github.com/ipython/ipython/blob/master/IPython/displ...
ipython.core.display: https://github.com/ipython/ipython/blob/master/IPython/core/...
ipython.lib.display: https://github.com/ipython/ipython/blob/master/IPython/lib/d...
You can also run Jupyter kernels in a shell with jupyter/jupyter_console:
pip install jupyter-console jupyter-client
jupyter kernelspec list
jupyter console --kernel python3Ask HN: What is the cost to launch a SaaS business MVP
I interviewed several entrepreneurs, I noticed several spent so much money on developing their product before launch. What is your experience?
When you're not yet paying yourself, your costs are your living costs and opportunity costs (in addition to the given fixed and variable dev and prod deployment cloud costs).
Early feedback from actual customers on an MVP can save lots of development time. GitLab Service Desk is one way to handle emails as issues from users who don't have GitLab accounts.
A beta invite program / mailing list signup page costs very little to set up; you can start building your funnel while you're developing the product.
Cryptocurreny crime is way ahead of regulators and law enforcement
I used to be surprised when I would read articles about this mythical legislative process which can and will protect lowly plebians like me. I'm not sure why so many people are under the impression legislation will be able to keep up with perversely motivated individuals or groups. Who are these legislators with motives so pure that they will never be bribed to only look out for my interests? Let's never forget there is only one person in jail for the 2008 crash this article is trying to convince us was the result of too little regulation. There was never enough incentive to protect us from the people who created that market crash.
I think the problem with 2008 was that no one was doing anything illegal, just a lot of poor judgement and lack of regulations.
Bitcoin was created by people who were so peeved at the regulator's interventions in the '08 crisis that they decided that bypassing the entire financial system was the best option. Forever immortalising "The Times 03/Jan/2009 Chancellor on brink of second bailout for banks" just as a reminder of why they were doing what they did [0].
We don't need better regulation. We need, when financial firms muck up on a global scale, to replace the people who run those firms with different people. There is an easy way to do that, let them go broke.
The biggest problem in the room is that the regulators have bought in to the idea that some companies are "too big to fail". That is a stupid idea, fundamentally unfair, completely throwing out the best part of capitalism which is that recognised idiots aren't allowed to be in charge. That problem will certainly not be solved by giving the regulators more power.
Bitcoin was created in context to "Transparency and Accountability": a campaign motto not coincidentally found in the title of the "Federal Funding Accountability and Transparency Act of 2006".
> The Federal Funding Accountability and Transparency Act of 2006 (S. 2590)[2] is an Act of Congress that requires the full disclosure to the public of all entities or organizations receiving federal funds beginning in fiscal year (FY) 2007. The website USAspending.gov opened in December 2007 as a result of the act
Sen. Obama's office is the origin of this bill; which was fronted by Sens Coburn and McCain, who had the clout.
https://usaspending.gov/ creates a mandatory database with budgetary line item metadata. Where money actually goes is something that is far more transparent and accountable with bitcoin and other public ledgers than any existing ledger covered by bank secrecy laws.
For context, in 2008-09, global financial systems were failing as a result of the American economy: housing bubble burst, HFT "flash crash" that we didn't have CAT or big data tools to determine the cause of, DDOS attacks and cyber security losses increasing YoY, credit default swaps had been rated as AAA securities (they sold bundled bad debt like it was worth something, and then wrote down losses), Enron energy speculation amidst rolling blackouts that were leaving hospitals in the dark, on gas generators, government investments in renewables had been paltry since the Carter administration had put solar panels on the roof of the White House before the whole oil price shock, and oil commodity speculation had driven the price of oil to like 2-4x the 2000 price (with resultant price effects on most CPI inflation/PPP basket goods); but electricity consumption was down in 2008 and renewables hadn't reached production volumes necessary to reach the competitive price point that renewables now present: cheaper than nonrenewables.
Who would have thought that the speculative price would continue to exceed the production cost. incentives or penalties?
Externalities per dollar returned per kWh is one way to assess the total costs of electricity production methods.
"Buy Gold" was the refrain of the day: TV commercials, signs out in front of piano stores (a somewhat-arbitrary commodity, sales of which are observed to be a leading indicator of economic health), signs on the road. And the message was "take your money out the market and put it in gold" which drives up the prices for chips and boards and medical equipment that rely upon that commodity as a material input. Gold is necessary for tropical spec components in high-humidity environments: gold hinges are prized, for example.
But, "look, there's water flowing from the chocolate fountain; so you can go ahead and go" and "you know you want to put it back in there, in that market" we're the appropriate messages given our revenue at the time.
For further technology scene context in 2007-2009,
"Grid computing" links to a number of distributed computing projects: https://en.wikipedia.org/wiki/Grid_computing#History
IIRC, there was a production metric-priced grid system developed around Seattle/Vancouver called "Gold" (?) that was built on Xen and is likely a precursor to metric-priced Cloud services like EC2 and S3 (which now simplify calculations for how much a 51% attack against a Proof of Work txpool with adaptive tx fees costs with n good participants in the game) which incentivize efficiency by penalizing expensive operations.
Code bloat was already a thing: how is everything getting slower when Moore's Law predicts the growth rate in transistor density? Are there sufficient incentives for code efficiency when there seem to be surplus compute resources just idly depreciating.
MySQL primary/secondary replication was considered a viable distributed database system, but securing replication depends upon cert exchange and (optionally), PKI, DNS, and IP tunnels of some sort. And then who has root, write to the journal and tables and indexes on the filesystem, UPDATE, and DELETE access in an inter-organizational distributed systems architecture with XML, Web Services, our very own ESB to scale separately from the database replication and off-site backups that nobody ever checks against the online data, and fragmented and varyingly-implemented industry standards that hopefully specify at least a sufficient key for the record that's unique across ledgers/systems/databases.
BitTorrent DHT magnet: links were extant.
Linden Dollars in Linden Lab's Second Life (there's a price floor on land, which is necessary to sell digital assets/goods/products/services) and accumulated avatar value in e.g. EverQuest and WarCraft (for which there were secondhand markets).
ACH was ACH: GPG-signed files over SFTP on the honor of the audited bank to not allow transfers that deposit money that doesn't exist.
There was no common struct for banking APIs (as apparently only e.g. Plaid, Quicken, and Mint solve for): ledger transactions have a fixed width text field that may contain multiple fields concatenated into one string, and there's no "payee URI" column in the QIF or CSV dumps of an account ledger.
To request more than e.g. the past 90 days of one's own checking account ledger, one was expected to parse tables out of per-month PDFs with e.g. PDFminer at $20 apiece, and then think up ones own natural key in order to merge and lookup records because (2008-01-01,3.99) and (2008-01-01,3.99,storename) are indistinct as a natural key (and when hashed). If you loan a your bank money (for them to now freely invest in the other side, since GLBA in 1999), wouldn't you think that the least they can do is give you `SELECT * WHERE account_id=?` as a free CSV without any datetime limitation in regards to what's offline and what's online.
"Audit the Fed", "Audit DoD" were being chanted by economically-aware citizens amidst severe correction and what was then the most severe recession since the Great Depression: the "Great Recession" it was called, and payouts to essential cronies (who hadn't saved wheat for the famine) were essential.
Overdraft was an error charged to the customer, who didn't build an inconsistent system (CAP theorem) that allows spending money that doesn't exist (at interest charged to the consumer/taxpayer).
"Catch Me If You Can" (2002) described the controls for bank fraud at the time. Why are fees so high?
"Office Space" (1999) described penny-shabing / salami-slicing attack: "fractions of a penny".
"Beverly Hills Ninja" (1997) detailed the story of the Great White Ninja and Tanley! (fistpalm)
"Swordfish" (1999) described a domestic disaster and bank transfers confirmable in seconds.
Ask HN: Why aren't micropayments a thing?
Amazon aws and related services can charge you a rate per email, or per unit time of computation, so why can't news sites just charge you $0.01 to read an article, or even half that?
For reference, there's Web Monetization [1] which tries to solve exactly that.
As others have noted it all boils down to user agent support. Otherwise most publishers probably won't consider giving up ad or subscription financed models.
Also, forcing users into subscriptions allows for better demographics data/statistics.
https://webmonetization.org/ lists Coil (flat $5/mo) as the first Web Monetization provider: https://coil.com/
Web Monetization builds upon ILP (Interledger Protocol), which is designed to work with any type of ledger; though it's probably not possible for any traditional ledger to beat the <1¢ transaction fee that only pre-mined coins have been able to achieve.
I really like their approach and I paid for Flattr for quite a while when it started. But something big needs to push them to the critical adoption rate. On a number of very techy blog posts, I got... 0 from WebMonetization. And that's on best case audience (coming from HN and tech Reddit).
Elon Musk announces $100M carbon capture prize
https://www.xprize.org/prizes/carbon :
> The $20M NRG COSIA Carbon XPRIZE develops breakthrough technologies to convert CO₂ emissions into usable products.
CCS: https://en.wikipedia.org/wiki/Carbon_capture_and_storage
CCU: https://en.wikipedia.org/wiki/Carbon_capture_and_utilization
Sequestration: https://en.wikipedia.org/wiki/Carbon_sequestration
Tim Berners-Lee wants to put people in control of their personal data
Interesting in a technological sense, but what problem it solves isn’t obvious to me. It lets me granularly authorize first party access what data I have in my pod, but there can’t be any technical guarantees with regards to illegitimate sharing or otherwise copying (many might at least cache, for instance) – nor about what is collected and shared outside this system.
I keep seeing data-hubs and identity-providers touting themselves as solutions to the web's privacy issues, but I don't see how they actually solve anything.
It seems like an attempted technical solution to a social problem to me.
The real problem with data based services (ads, Google search, etc) is really that a bunch of data is collected opaquely, unethically, and in some cases illegally. The whole system including data brokers and real time bidding is out of control.
In my (personal) view, it's the technical part of a solution that definitely also needs to have a social/legislative component. It cannot prevent parties from illegitimate sharing of my data, but it does give them the option to hand over control to me. There are lots of companies that currently hold data on us but for whom that data is not their primary competency, and they only need a small nudge (like GDPR) to make having the customer responsible for that data an attractive proposition.
You might be right, but I think it's disingenuous to market it as though this "solves privacy". Worst case, people are lulled into a false sense of security.
Data-storage + authorization doesn't solve any (new) technical privacy-issues; this is "data protection" rather than "data privacy" in my book.
While I recognize the value of W3C LDP and SOLID, I also fail to see anything in SOLID that prevents B from sharing A's now pod-siloed information.
Does it prevent screenshots and OCR?
So it's in standard record structs and that makes it harder for the bad guys?
Who moderates mean memes with my face on them?
It is my hope that future Linked Data spec tutorials model something benign like shapes or cells instead of people: so that we can still see the value.
Laws still exist against things like perjury, even though the existence of the law is not a technical means in itself able to prevent perjury. Note that one of the comments upthread specifically mentioned legislation. The current notion that many people in the tech world have, which roughly states that what determines whether something is kosher is whether it's technically possible to accomplish, is something that needs to change, instead of things just staying a permanent Wild West forever.
There's also an old phrase that putting locks on your doors doesn't actually stop a determined attacker, but that it's okay because they're not meant to—that they're meant to "keep honest people honest". It's a principle that applies here.
No, there are few to no actual privacy improvements over centralized systems.
Perhaps even functional regression: what, are you going to run a hash blocklist across all nodes? Like spamhaus? Is there logging or user accounting? Is anything chain of custody admissable, or are we actually talking about privacy and liberty here?
Is everything just marked, "not for unlimited distribution"? And we dwpend upon there not being bad actors?
Real costs are very different with just friendly early adopters.
Cryptographically signing posts (with LD-Signatures) may help with integrity, but that can be done with centralized systems and does nothing to help with confidentiality.
What about availability? Is it a trivially-DOS'able system?
Governments spurred the rise of solar power
I can't read the article, but I think evidence like this is important when it comes to the whole idea of government subsidises impacting on free markets. The free market has become a bit of a religion for some (you can see in the comments here), and there's this staunch divide between those that believe the free market will always come up with a humanitarian solution, to those that believe the government need to push things for it to happen.
My FIL was head of a large coal plant in the UK, and is very critical of renewable energies, their cost, the impact on the fossil fuel industry etc... It simply isn't possible to have a conversation with him about global warming - he will simultaneously argue it isn't happening and it's natural and not caused by us. He might be an extreme example, but there is general scepticism of government subsidies here.
I'd love to see a study where researchers take things that are seen as good/important/essential to modern life and measure the amount of public/government sponsorship that helped bring it about.
I always find the free market vs government arguments a false dichotomy. Even the free-est markets require a whole host of government services: regulation, standards setting, contracts enforcement, antifraud measures provision and infrastructure to name but a few. People only cry free market when they don't like a particular government activity.
That coal plant relies on infrastructure to get its coal. It needs fair utility prices to deliver its power to customers. It needs reliable (abuse free) markets to sell the power on. It needs voltage and frequency/time standards to access the network. It needs contract enforcement to actually get paid by its customers. It needs insurance and educated workers and a CEO who won't embezzle.
All provided by government, all welcomed by industry. But suddenly government has no place in free markets when it comes to CO2.
Should we prefer penalties or incentives in order to use predictable markets for the change we need?
That depends on whether you want power to be cheaper or more expensive. Cheaper is nice but you'll pay for it taxes etc. Expensive is painful at the point of consumption but encourages efficiency. It's a strategic and political decision and depends on a nations needs and objectives...
> I'd love to see a study where researchers take things that are seen as good/important/essential to modern life and measure the amount of public/government sponsorship that helped bring it about.
Essential technology investment of US tax dollars?
NASA spinoff technologies: https://en.wikipedia.org/wiki/NASA_spinoff_technologies
NSF, DARPA, IARPA, In-Q-Tel, ARPA-e (2009)
List of emerging technologies: https://en.wikipedia.org/wiki/List_of_emerging_technologies
Termux no longer updated on Google Play
The GitHub discussion is significantly more informative and carries a lot of thinking behind the changes: https://github.com/termux/termux-app/issues/1072
IMO a better link than a short paragraph on Wiki.
Note that there are 297 hidden items in that issue so you have to click "Load more..." ceil(297/60) times to read all of the comments about how APK packaging is soon necessary for latest Android devices so the termux package manager can't just dump executable binaries wherever.
FWIU:
- Android Q+ disallows exec() on anything in $HOME, which is where termux installed binaries that may have been writeable by the executing user.
- Binaries installed from APKs can be exec()'d, so termux must keep APK repacks rebuilt and uploaded to a play store.
- Termux shouldn't be installed from Google Play anymore: you should install termux from the F-Droid APK package repos, and it will install APKs instead of what it was doing.
- Compiling to WASM with e.g. emscripten or WASI was one considered alternative. "Emscripten & WASI & POSIX" https://github.com/emscripten-core/emscripten/issues/9479
What about development on-the-device?
- It seems C compiled with clang on the device wouldn't be executable? (If it was, that would be a way around the restriction: distribute packages as source, like the good old days)
> offer users the option of generating an apk wrapping their native code in a usable way. https://github.com/termux/termux-app/issues/1072#issuecommen...
This seems a promising solution: compile from source, create an apk, install - your custom distribution! For popular collections of packages, a pre-built apk.
- Java might be explicitly blocked, being a system language for android, even though its byecode is interpeted and not exec()ed.
- Other interpreted languages should be OK e.g. python
> > offer users the option of generating an apk wrapping their native code in a usable way.
> This seems a promising solution: compile from source, create an apk, install - your custom distribution! For popular collections of packages, a pre-built apk.
FPM could probably generate APKs in addition to the source archive and package types that it already supports.
The conda-forge package CI flow is excellent. There's a bot that sends a Pull Request to update the version number in the conda package feedstock meta.yml when it detects that e.g. there's a new version of the source package on e.g. PyPI. When a PR is merged, conda-forge builds on Win/Mac/Lin and publishes the package to the conda-forge package channel (`conda install -y -c conda-forge jupyterlab pandas`)
The Fedora GitOps package workflow is great too, but bugzilla isn't Markdown by default.
Keeping those APKs updated and rebuilt is work.
Ask HN: What should go in an Excel-to-Python equivalent of a couch-to-5k?
Yesterday, my co-founder published a blog about her experiences Ditching Excel for Python in her job as a Reinsurance Analyst [0].
One of the responses on reddit [1] asked what they should do, "Step 1 day 1," if having read Amy's post they were convinced to try and begin the long journey from tangled Excel/Access spaghetti.
My (flippant) reaction to a friend that brought the comment to my attention was unhelpful; "Step 1 day 1, quit." So he has challenged me to write eight helpful blog posts during the remainder of my Garden Leave.
What should go in them?
[0] https://amypeniston.com/ditching-excel-for-python/
How to write functions in JS / VB script and call them from a cell expression.
How to name variables something other than AB3.
How to use physical units and datatypes. (How to specify XSD datatype URIs that map to native primitives in an additional frozen header row. e.g. py-moneyed and Pint & NumPy ufuncs)
How transitive sort works (is there a tsort to determine what to calculate first (and whether there are cycles) on every modification event?)
Which Jupyter platforms do and don't support interactive charts with e.g. ipywidgets?
pandas.df.plot(kind=) (matplotlib), seaborne (what are the calculated parameters of this chart?), holoviews, plotly, altair
Reproducibility w/ repo2docker / BinderHub:
pip freeze > constraints.txt
cp constraints.txt requirements.txt
conda env export --from-historyAlso,
When is it better to have code in a notebook instead of in a module?
How to export notebook cells to a module with nbdev
How to write tests to assert the quality of the code and the model: @pytest.mark.parametrize, pytest-notebook, jupyter-pytest-2, pytest-jupyter
When is it appropriate to parametrized a notebook with e.g. papermill?
How to handle concurrency: dask.distributed + dask-labextension, ipyparallel
Interesting, your suggestion is that the first thing to do is to learn how to make functions and variable names within excel?
Yeah if you port it to functions and verify that you haven't broken anything, you could then easily port to Python functions that you could call from Excel with an add-on that everyone that opens the sheet needs to have installed; but everyone that opens a sheet that calls Python must have that same extension (and all python package dependencies) installed, too
>My (flippant) reaction to a friend that brought the comment to my attention was unhelpful; "Step 1 day 1, quit." So he has challenged me to write eight helpful blog posts during the remainder of my Garden Leave.
One way to do it would be to do impedance matching: maximizing usefulness by bringing Python to the universe the person who asked the question lives in: Excel, Word, PDF, email.
This means "Automate the Boring Stuff"[0] to ease into programming with something specific to do that is relevant and useful right off the bat for manipulating files, spreadsheets, PDF documents, etc.
Openpyxl[1] for playing more with Excel files, and basically searching "Excel Python" on the internet for more use cases.
Python-docx[2] for playing with Word documents, extracting content from tables, etc.
PdfPlumber[3] for playing with PDFs. Sometimes you have data in .pptx files, and you can use LibreOffice to convert all of them to PDFs because you hate the pptx format:
libreoffice --headless --invisible --convert-to pdf *.pptx
- [0]: https://nostarch.com/automatestuff2
- [1]: https://openpyxl.readthedocs.io/en/stable/
- [2]: https://python-docx.readthedocs.io/en/latest/
Using openpyxl to extract data from exist excel workbooks and other documents would seem like a good place to start.
I suspect many companies use excel workbooks as "forms" with lots of data at the same cell in multiple workbooks.
> I suspect many companies use excel workbooks as "forms" with lots of data at the same cell in multiple workbooks.
Downstream data quality costs can be minimized with data normalized schema and data collection process controls like forms-based data validation.
There are established UI/UX design patterns for data validation of user-supplied data: accessible [web] forms with tab-ordered input fields and specific per-input feedback with accessible HTML5 and ARIA. IIUC, Firefox now supports PDF forms, too?
Why would we move from a spreadsheet to an actual database?
Data integrity:
Referential integrity (making sure that record keys actually point to something when creating, updating, or deleting),
Columnar datatypes (float, decimal, USD, complex fraction),
Access controls (auth(z): authentication and authorization),
Auditing (what was the value before and after that) and Disaster Recovery,
Organizationally-unified schema development and corresponding validation.
Repeatability / Reproducibility: can you replay the steps needed to build the whole sheet? What parameters were entered and how to we script that par so that we can easily assess the relations between the terms of the argument presented?
Scientists turn CO2 into jet fuel
Link to the article in question: https://www.nature.com/articles/s41467-020-20214-z
They call it 'renewable' yet use Iron as a catalyst. I'm not a chemist - how is that renewable?
Yao, B., Xiao, T., Makgae, O.A. et al. "Transforming carbon dioxide into jet fuel using an organic combustion-synthesized Fe-Mn-K catalyst." Nat Commun 11, 6395 (2020). https://doi.org/10.1038/s41467-020-20214-z
Synthetic biology can do it better. Fix CO2 by growing sugar cane, turn sugar into jet fuel with genetically engineered yeast.
https://www.total.com/media/news/press-releases/total-and-am...
Still not commercially viable, but much closer to it than the linked process.
That approach requires lots of land, water, and time. Nothing wrong with that, but much of the funding for CO2->jet fuel research comes from the military, which wants to be able to create liquid fuel quickly from CO2 in the atmosphere and they don't care how inefficient it is. Military customers are typically not very interested in processes that require running a farm.
The military also spends a lot of time turning jet fuel into electricity, because the former is energy-dense, easy to store, easy to transport. And they need electricity in lots of places without a grid.
Where would the reverse be useful? Somehow you have unlimited electricity but no fuel, and you have time and space to run a chemical refinery? (And isn't carbon capture from the atmosphere likely to require farm-scale infrastructure anyway?)
You cannot run aircraft on electricity; liquid fuel is the only option. The military is keenly interested in being able to refuel aircraft in situations where liquid fuel shipments might be denied or unavailable for some reason.
You can run aircraft on electricity.
Locomotives run on electrical energy produced by diesel generators (because electric motors are more energy efficient), for example.
The limits are the cost and weight of the batteries and the charge time.
You can't in any practical sense. Locomotives don't have a problem carrying the weight of a diesel engine plus a generator; in fact the extra weight works in their favor.
If you tried that trick with an airplane it would never get off the ground. Locomotives use this system not for efficiency but because they need the low-end torque electric motors provide. (If they only needed efficiency they'd just drive the wheels with the diesel engine directly.) Airplanes don't need torque; they need power.
You can run an airplane on batteries but only for a few minutes with modern battery technology. Practical battery-powered aircraft for military applications are not going to happen any time soon. We might see battery-powered air taxis soon but those are not even close to meeting military requirements for fighters, transports, etc.
From https://en.wikipedia.org/wiki/Electric_aircraft :
> [For] large passenger aircraft, an improvement of the energy density by a factor 20 compared to li-ion batteries would be required
The time it takes to surpass this energy density threshold is affected by battery tech investments; which had been comparatively paltry in terms of defense spending. Trillions on batteries would've been a much better investment; with ROI.
Sadly, some folks in defense still can't understand why non-oil investments in battery tech are best for all.
There are multiple electric trainer aircraft with flight times over an hour and quite a few more in development.
Jet engines are terribly inefficient (30-50% efficient) compared to electric motors.
Show HN: Stork: A customizable, WASM-powered full-text search plugin for the web
OP - congratulations on shipping! WASM-powered search caught my eye.
Looks like the search index is downloaded and used locally in the browser, so this is as fast as search can get. One trade-off though is that you're limited to relatively small datasets. While this shouldn't be an issue for small-medium static sites, an index that needs to be downloaded over the wire will affect your page performance for larger sites / datasets.
In those cases, you'd want to use a dedicated search server like Algolia or its open source alternative Typesense [1]. Both of them store the index server-side and only return the relevant records that are searched for.
For eg: you'd probably not want to download 2M records over the wire to search through them locally [2]. You'd be better off storing the index server-side.
[1] https://github.com/typesense/typesense (shameless plug)
Just a quick non-thought-through idea but would it be possible to build an index in a way that allows clients to download only parts of it based on what they search? I.e. the search client normalizes the query in some way and then requests only the relevant index pages. The index would probably be a lot larger on the server but if disk space is not an issue...?
(Though at some point you have to ask yourself what the benefits of such an approach are compared to a search server.)
Merkle Search Trees: Efficient State-Based CRDTs in Open Networks https://hal.inria.fr/hal-02303490/document
Peer-to-Peer Ordered Search Indexes https://0fps.net/2020/12/19/peer-to-peer-ordered-search-inde... (which adds useful context about the above)
Sorry for just dropping these links, I should already be asleep :)
> Merkle Search Trees: Efficient State-Based CRDTs in Open Networks https://hal.inria.fr/hal-02303490/document
https://scholar.google.com/scholar?cites=7160577141569533185... ... "Merkle Hash Grids Instead of Merkle Trees" (2020) https://scholar.google.com/scholar?cluster=13503894708682701...
Browser-side "Blockchain Certificate Transparency" applications need to support at least exact key lookup by domain/SAN and then also by cert fingerprint value; but the whole CT chain with every cert issue and revocation event is impractically large in terms of disk space.
https://github.com/amark/gun#history may also be practically useful.
Upptime – GitHub-powered open-source uptime monitor and status page
Maker here, I made Upptime as a way to scratch my own itch... I wanted a nice status page for Koj (https://status.koj.co) and was previously using Uptime Robot which didn't allow much customization to the website.
It started with the idea of using GitHub issues for incident reports and using the GitHub API to populate the status page. The obvious next step was opening the issues automatically, so the uptime monitor was born. I love using GitHub Actions in interesting ways (https://github.blog/2020-08-13-github-actions-karuna/) and have made it a large part of my toolkit!
https://news.ycombinator.com/item?id=25557032 mentions "~3000 minutes per month". GitLab's new pricing structure: [(runner_minutes, usd_per_month), (400, $0), (2_000, $4), (10_000, $19), (50_000, $99)]
You can run a self-hosted GitHub or GitLab Runner with your own resources: https://docs.github.com/en/free-pro-team@latest/actions/host...
GitLab [Runner] also runs tasks on cron schedules.
The process invocation overhead for CI is greater than for a typical metrics collection process like a nagios check or a memory-resident daemon like collectd with the curl plugin and the "Write HTTP" plugin (if you're not into using a space and time efficient timeseries database for metrics storage)
An open source project with a $5/mo VPS could run collectd in a container with a config file far far more energy efficiently than this approach.
Collectd curl statistics: https://collectd.org/documentation/manpages/collectd.conf.5....
Collect list of plugins: https://collectd.org/wiki/index.php/Table_of_Plugins
Is there a good way to do {DNS, curl HTTP, curl JSON} stats with Prometheus (instead of e.g. collectd as a minimal approach)?
Show HN: Simple-graph – a graph database in SQLite
I wonder if there are ways, in SQLite, to build indices for s,p,o/s,p/p,o/ and maybe more subtle ones... That would be uber nice, given the fact that most graph databases have their own indexing strategies, and you cannot craft your own.
rdflib-sqlalchemy is a SQLAlchemy rdflib graph store backend: https://github.com/RDFLib/rdflib-sqlalchemy
It also persists namespace mappings so that e.g. schema:Thing expands to http://schema.org/Thing
The table schema and indices are defined in rdflib_sqlalchemy/tables.py: https://github.com/RDFLib/rdflib-sqlalchemy/blob/develop/rdf...
You can execute SPARQL queries against SQL, but most native triplestores will have a better query plan and/or better performance.
Apache Rya, for example:
> indexes SPO, POS, and OSP.
Thanks for your comment. I use rdflib frequently but have never tried the SQLAlchemy back end. Now I will. That said, Jena or Fuseki, or the commercial RDF stores like GraphDB, Stardog, and Allegrograph are so much more efficient.
In CPython, types implemented in C are part of the type tree
The docs should have coverage on this:
Python/C API Reference Manual: https://docs.python.org/3/c-api/index.html
Python/C API Reference Manual » Object Implementation Support > Type Objects: https://docs.python.org/3/c-api/typeobj.html
CPython Devguide > Exploring Python Internals > Additional References: https://devguide.python.org/exploring/
Experiments on a $50 DIY air purifier that takes 30s to assemble
From "Better Box Fan Air Purifier" https://tombuildsstuff.blogspot.com/2013/06/better-box-fan-a... :
> Air purifiers can be expensive and you've probably seen articles recommending to just put a 20" x 20" x 1" furnace filter on a cheap 20" box fan and POOF! instant cleaner air for not a lot of money. It really does clean the air pretty cheap.
> There's a problem with this though. These fans weren't designed to be run with a filter. The filter will restrict air flow which will put a higher strain on the motor causing it to use more electricity and in worse cases could be a fire hazard. The higher the MERV rating (cleaning efficiency) of the filter the more stress it will put on the fan.
> Don't worry! You can still have your cheap air purifier as long as the filter area is increased to decrease the effect of air resistance. Instead of using one 20x20x1 filter we'll use two 20x25x1 filters which increases the filter surface area over 250%. It's a little more expensive because you're using two filters instead of one but the increased filter surface area also helps the filter last longer before it gets clogged up and we're saving on energy use compared to a single filter.
> There's a problem with this though. These fans weren't designed to be run with a filter. The filter will restrict air flow which will put a higher strain on the motor causing it to use more electricity and in worse cases could be a fire hazard. The higher the MERV rating (cleaning efficiency) of the filter the more stress it will put on the fan.
I'm fairly certain the opposite is true. In fact, most rowing machines work on this principle. The easiest setting is the one where the fan is as closed off as possible because it's pulling a vacuum. Less air, less resistance, easier to row / less power required to spin the fan. If anything, fans should draw less current with air flow on the inlet side restricted.
A. Putting two [larger] filters in a 'V' with cardboard to fill the top and bottom pulls the same amount of air through a larger area of filters
B. pulling the same volume of air through greater surface area results in greater pressure between the filter and fan than one filter directly affixed to the fan
C. The lower air pressure / "suction" due to an obstructed intake causes an electric fan motor to fail more quickly.
D. Increasing the air pressure that the motor is in reduces the failure rate?
Goodreads plans to retire API access, disables existing API keys
This makes absolutely no sense and has no relation to any economic variables. Goodreads isn’t some struggling self-funded startup — it’s owned by Amazon.com. The acquisition was a deal that should have never been approved, if the Obama administration had been anything beyond completely impotent at protecting us from monopoly games:
https://www.theguardian.com/books/2013/apr/02/amazon-purchas...
I would like to understand the true strategic interest behind this. Is Amazon simply penny-pinching now that they’ve successfully obliterated the market for both new and used books online? There’s way more to this story than appears on the surface.
It's not mentioned in the article but Amazon had disabled their affiliate link program for Goodreads ahead of this announcement, which cut off a major source of their revenue for them, and forced them to sell. They had no choice.
The strategic reason for Amazon is obvious. As someone else mentioned, Amazon doesn't want Goodreads data to be used to add value to their competitors' offerings.
Speaking as a developer who tried to build on top of Goodreads API, I also want to add that this was a long time coming. The API had been neglected for some time. And some of the most interesting datasets weren't even made available through the API.
Apparently Amazon’s Kindle lost the ability to share progress on Goodreads in the last week as well.
I guess the writings on the wall...
Turing Tumble Simulator
(Turing Tumble is a (fun) marble-powered computer game.)
Python Pip 20.3 Released with new resolver
If anyone hasn't seen it, now is a good time to look at https://python-poetry.org/ It is rapidly becoming _the_ package manager to use. I've used it in a bunch of personal and professional projects with zero issues. It's been rock solid so far, and I'm definitely a massive fan.
I occasionally ask our principle / sr. Python engineers about this, and their response is always, "These things come and go, virtualenv/wrappers + pip + requirements.txt works fine - no need to look at anything else."
We've got about 15 repos, with the largest repo containing about 1575 files and 34MBytes of .py source, 14 current developers (with about 40 over the last 10 years) - and they really are quite proficient, but haven't demonstrated any interest at looking at anything outside pip/virtualenv.
Is there a reason to look at poetry if you've got the pip/virtualenv combination working fine?
People who use poetry seem to love it - so I'm interested in whether it provides any new abilities / flexibility that pip doesn't.
beware of zealot geeks bearing gifts. if your environment is currently working fine and you are only interested in running one version of python and perhaps experimenting with a later one then venv + pip is all you need, with some wrapper scripts as you say to make it ergonomic (to set the PYTHON* environment variables for your project, for example)
The main reason I use Poetry is for its lockfile support. I've written about why lockfiles are good here: https://myers.io/2019/01/13/what-is-the-purpose-of-a-lock-fi...
Pip supports constraints files. https://pip.pypa.io/en/stable/user_guide/#constraints-files :
> Constraints files are requirements files that only control which version of a requirement is installed, not whether it is installed or not. Their syntax and contents is nearly identical to Requirements Files. There is one key difference: Including a package in a constraints file does not trigger installation of the package.
> Use a constraints file like so:
python -m pip install -c constraints.txtWhat does that command do? Does it install dependencies, or simply verify the versions of the installed dependencies?
It just gives you an error:
ERROR: You must give at least one requirement to install (see "pip help install")
Your package manager should be boring, extremely backward and forward compatible, and never broken. Experience has shown this not to be true for python. Several times over the years i’ve found myself, pinning, upgrading, downgrading, or otherwise juggling versions of setuptools and pip in order to work around some bug. Historically I have had far more problems with the machinery to install python packages I have had with all of the other python packages being installed combined, and that is absurd.
Convolution Is Fancy Multiplication
FWIW, (bounded) Conway's Game of Life can be efficiently implemented as a convolution of the board state: https://gist.github.com/mikelane/89c580b7764f04cf73b32bf4e94...
How to better ventilate your home
I recently purchased a home and spent some time improving ventilation in it this year. This is a home in Minneapolis, so just "opening a window" is not an option, as it would be too cold in the winter and dramatically reduce energy efficiency. Its effectiveness also changes based on factors like the wind, and the temperature difference between inside and out.
The general strategy for doing this right now with existing homes is to insulate your home as well as possible for energy efficiency, then install a continuous ventilation fan. This is essentially a bathroom fan, except that it runs all the time at a constant low speed, helping to circulate the air through and out your house by "pulling" a designed amount of air through the cracks.
I didn't want to punch a new hole in the house just to do this, and already had a bathroom fan installed, so as a hack I just turned it into a whole house ventilation fan with this: https://www.aircycler.com/pages/smartexhaust
Basically you calculate how much CFM you need per hour based on the square footage of your house, and then you set it on the fan control. It still acts like a bathroom fan, except every hour it also runs for a set period of time (in my case, about 12 minutes).
The standard for this is ASHRAE 62.2. Use this to calculate the CFM for code: https://homes.lbl.gov/ventilate-right/step-3-whole-building-...
Then the formula for calculating the fan run time is on this sheet: https://cdn.shopify.com/s/files/1/0221/7316/files/AC_DOC_7_0...
And presto, you have a ventilation system without having to do a lot of work. Do _not_ try this with a crappy, rusty old bathroom fan - clean or replace the motor first, and for extra credit, use an arc fault circuit interrupter on the breaker so if the motor fails it will blow the fuse instead of potentially causing a fire.
Note: This is really just to manage general air quality and VOCs. If you want to specifically make ventilation for COVID-19, that's a different problem. They focus on Air Changes per Hour (ACH), and a cubic feet calculation is used rather than square feet. There's no "recommended amount" of ACH for managing COVID-19. You're likely improving the situation by increasing it, but I wouldn't start inviting people over after you did it. ACH is very high in ICUs but staff are still getting sick there.
RE Humidity - ideal range varies based on region and outside temperature, but this chart roughly shows it: https://lh3.googleusercontent.com/proxy/mPz-jGdLpnPGgvWR3Egf...
I have an on-furnace humidifier controlled by an ecobee for the winter, it's a huge quality of life improvement if you live in cold climates, but make sure to set it to "frost control" otherwise it won't lower the humidity based on the outside air and you can get mold in your walls. For summer, a standard house A/C combined with continuous ventilation should be sufficient to bring down humidity levels.
Finally, the "correct" air ventilation is a moving target with trade-offs and concerns of the moment. It was higher in 1925 (30cfm/person) to try to prevent tuberculosis and infectious diseases, then was lowered to 5cfm/person in the 70s during the energy crisis, and is currently at 15cfm/person. I imagine COVID-19 could make us re-consider the current recommendations. https://homes.lbl.gov/ventilate-right/ashrae-standard-622
I'm in an older home that leaks a sieve but has central air. Would turning on that house fan to circulate without running the AC accomplish the same thing? Also, do any smart thermostats allow this sort of thing to be automated?
"Fan control with a Nest thermostat" https://support.google.com/googlenest/answer/9296419?hl=en
Looks like there could be: (1) an every hour for n minutes schedule; (2) an option to run the fan with the thermostat off; (3) an option to shut off the fan when everyone is gone
Quantum-computing pioneer Peter Shor warns of complacency over Internet security
If an organization has a 5 year refresh cycle (~time to implement a new IT system), and there exists a quantum computer with a sufficient number of error-corrected qubits by 2027 [1], an organization/industry has 5 years from 2022 to go quantum-resistant: replace their existing solution with quantum-resistant algos (and, in some cases, a DLT with a coherent pan-industry API) and/or double their RSA and ECDSA key sizes.
[1] "Quantum attacks on Bitcoin, and how to protect against them (ECDSA, SHA256)" https://news.ycombinator.com/item?id=15907523
Which DLT/blockchains without PKI (or DNS) will implement the algorithms selected from the NIST Post-Quantum Cryptography (PQC) round 3 candidate algorithms? https://csrc.nist.gov/projects/post-quantum-cryptography
CERN Online introductory lectures on quantum computing from 6 November
This is probably a dumb question but are there any data sciencey or tensorflowy things that can be done faster on a quantum computer?
There are some basic linear algebra subroutines (Matrix inversion, finding eigenvalues & eigenvectors) that can be performed with an exponential speedup on a quantum computer in theory, that's why there is so much interest in Quantum Machine Learning. If you are asking about the current hardware level, then no, current quantum computers can not solve any practical problem faster than a classical computer.
In theory classical computers are also not limited to have better solutions to problems where quantum computers claim superiority like prime factorisation or like Tang's quantum inspired classical algorithm that beats HHL for low rank matrices.
> There are some basic linear algebra subroutines (Matrix inversion, finding eigenvalues & eigenvectors) that can be performed with an exponential speedup on a quantum computer in theory
Eh... those particular subroutines have polynomial time algorithms already on a classical computer.
You can't exponentially speed up something that's polynomial time already.
https://quantumalgorithmzoo.org/ lists algorithms, speedups, and descriptions.
That's a great list, thanks!
(Though for the benefit of readers here, the list doesn't include any "basic linear algebra subroutines (Matrix inversion, finding eigenvalues & eigenvectors)").
A Manim Code Template
The demo video looks cool. It's maybe not obvious that there's a link to the code-video-generator (which is built on manim by 3blue1brown) demo video in the README: https://youtu.be/Jn7ZJ-OAM1g
Startup Financial Modeling: What is a Financial Model? (2016)
https://www.causal.app/ has free business model templates: SaaS (Foresight), eCommerce (https://foresight.is/), Startup Runway, Buy/Rent, Ads Calculator
The article explicitly, repeatedly says templates are a bad idea. If you disagree, it'd be interesting to see your reasoning, rather than links to free templates.
We'd do better to find a list of business modeling books and tools.
And then take a look at integrating actual data sources; hopefully some quantitative with APIs.
Uncertainties supports mean±"error" w/ "error propagation": https://pypi.org/project/uncertainties/
Sliders etc can be done in Jupyter notebooks with e.g. ipywidgets: https://ipywidgets.readthedocs.io/en/latest/
At what grade level do presidential candidates debate?
Intelligence does not imply superior moral, ethical, or rational judgement.
Incomplexity of speech does not imply lack of intelligence.
Here's the section on Simple English in Simple English Wikipedia: https://simple.wikipedia.org/wiki/Wikipedia:About#Simple_Eng...
Imagine being reprimanded for use of complex words and statistical terms in an evidence-based policy discussion in a boardroom. Imagine someone applying to be CEO, President, or Chairman of the Board and showing up without a laptop, any charts, or any data.
Topicality!
Perhaps there is a better game for assessing competency to practice evidence-based policy.
This commenter effectively refutes the claim that Fleisch-Kincaid is a useful metric for assessing the grade-level of interpretively-punctuated spoken language: https://news.ycombinator.com/item?id=24807610
Like I said, from "Ask HN: Recommendations for online essay grading systems?" https://news.ycombinator.com/item?id=22921064 :
> Who else remembers using the Flesch-Kincaid Grade Level metric in Word to evaluate school essays? https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readabi...
> Imagine my surprise when I learned that this metric is not one that was created for authors to maximize: reading ease for the widest audience is not an objective in some deparments, but a requirement.
> What metrics do and should online essay grading systems present? As continuous feedback to authors, or as final judgement?
That being said, disrespectful little b will not be tolerated or venerated by the other half of the curve.
ElectricityMap – Live CO₂ emissions of electricity production and consumption
What's going on in Queensland, Australia? "793g Carbon intensity" seems insanely high, especially as compared with Norway/France/Sweden (at 24-41g carbon intensity).
Does anyone have insight into this? I've never pictured Australia as a big contributor to pollution.
Good observation. I was curious, and it looks like the parser code to retrieve the data is open source; so here's the relevant module for Australia:
https://github.com/tmrowco/electricitymap-contrib/blob/24ea2...
The production mix (energy generation by source) source URL is listed on L326.
Opening the dataset in LibreOffice Calc, filtering by Region = QLD1, filtering for Current Output > 0, and sorting by Current Output descending does seem to show that the vast bulk of production reported has fuel source descriptor 'Black Coal'.
If that's a bug in the reported data or source, hopefully the ElectricityMap team can jump in to take a look. The data initially seems legit, from searching for some of the power stations listed.
Edit / addendum: worth searching for Australia in the project's GitHub issues too; there's some existing (although not directly related) investigation & validation work ongoing for that module
Also - home solar penetration in QLD is the highest in Australia. Is that not factored in to the equation? It doesn't look that way given 0% renewables shown.
This website is pulling information from public sources, so it can only show public generation & consumption information.
If someone is running a grid-tied solar system, outbound energy might be accounted for in these statistics, but it won't show any immediate consumption of that solar generation by the property, nor will it show any battery storage, it's only capable of showing what goes through the energy meter.
+1. California has substantial rooftop solar (11 Gwh peak) behind the meter, which isn’t shown in these graphs.
https://pv-magazine-usa.com/2019/04/15/californias-solar-pow...
How would behind the meter electricity consumption change the reported amount of CO2 emitted by other electricity production sources?
The map is intended to show the emissions intensity of electricity so if you miss a bunch of electricity from distributed generation you will over-estimate the intensity.
Man that was a dumb question. Thanks for clarifying.
So we'd need electric utility companies to share the live data of how many kwh of solar and wind people are selling back to the grid in order to get an accurate regional comparison of real-time carbon intensity?
FWIU, they're already parsing the EIA data; but it's significantly more delayed than the max 2 hour delay specified by ElectricityMap.
Here's the parser for the current data from EIA: https://github.com/tmrowco/electricitymap-contrib/blob/maste...
Should the EIA (1) source, aggregate, cache, and make more real-time data available; and (2) create a new data item for behind the meter kwh from e.g. residential wind and solar?
(Edit) "Does EIA publish data on peak or hourly electricity generation, demand, and prices?" https://www.eia.gov/tools/faqs/faq.php?id=100&t=3
> Hourly Electric Grid Monitor is a redesigned and enhanced version of the U.S. Electric System Operating Data tool. It incorporates two new data elements: hourly electricity generation by types of energy/fuel source and hourly sub-regional demand for certain balancing authorities in the Lower 48 states.
> [...]
> EIA does not publish hourly electricity price data, but it does publish wholesale electricity market information including daily volumes, high and low prices, and weighted-average prices on a biweekly basis.
AFAIU, retail intraday rates aren't yet really a thing in the US; but some countries in Europe do have intraday rates (which create incentives for the grid scale energy storage necessary for wide-scale rollout of renewables).
(Edit) "Introduction to the World of Electricity Trading" https://www.investopedia.com/articles/investing/042115/under... :
> Energy prices are influenced by a variety of factors that affect the supply and demand equilibrium. On the demand side, commonly referred to as a load, the main factors are economic activity, weather, and general efficiency of consumption. On the supply side, commonly referred to as generation, fuel prices and availability, construction costs and the fixed costs are the main drivers of the price of energy. There's a number of physical factors between supply and demand that affect the actual clearing price of electricity. Most of these factors are related to the transmission grid, the network of high voltage power lines and substations that ensure the safe and reliable transport of electricity from its generation to its consumption.
Which customers (e.g. data centers, mining firms) would take advantage of retail intraday rates?
How does cost and availability of storage affect the equilibrium price of electricity?
>So we'd need electric utility companies to share the live data of how many kwh of solar and wind people are selling back to the grid in order to get an accurate regional comparison of real-time carbon intensity?
You actually need production numbers for what's used "behind the meter" which by its nature is not directly available live or otherwise and has to be estimated.
>Which customers (e.g. data centers, mining firms) would take advantage of retail intraday rates?
Loads! In the UK you can get half-hourly priced electricity even at a domestic level (with a smart meter) and if you have loads that can be be re-scheduled (mainly EV charging but if you had an electric storage heater that would work as ell) you can save quite a lot of money.
Heavy industrial users definitely move usage around both to avoid expensive wholesale charges but also to reduce their transmission connection charges which (in the UK) are based on their usage during the most congested periods of the year. Water companies will vary pump operations for this reason and water pumping and treatment alone is 2% of electricity use.
Data centers definitely pay on a half-hourly settled basis but tend not to shift their workloads around to take advantage, some data centers will run their cooling systems in such a way as to reduce usage during the most expensive few half hours though. I have heard that larger users like Amazon, FB, Google, will automatically load balance between global centers to reduce electricity bills and carbon footprint.
> how many kwh of solar and wind people are selling back
Not just selling back. Producing - and then either using or selling.
This USA data seems kind of strange to me. For instance my home state, Kansas, we are fourth in the nation for installed wind capacity, supplying 41% of our energy demand. Our sole nuclear power plant provides an 18% base load. 33% is coal and the rest random crap. We're very population sparse and low population in general. Yet in other areas of the country that are strictly coal and natural gas fired, with a greater population density and higher total population, they're less turd colored. I'm guessing this is a strange interaction between renewable availability or it just might be a lack of available data.
From https://github.com/tmrowco/electricitymap-contrib#data-sourc... :
> Here are some of the ways you can contribute:
> Building a new parser, Fixing a broken parser, Changes to the frontend, Find data sources, Verify data sources, Translating electricitymap.org, Updating region capacities
I sent a few tweets and emails about the data in this region but nothing happened here either
Bash Error Handling
From https://twitter.com/b0rk/status/1312413117436104705 :
> TIL that you can use the "DEBUG" trap to step through a bash script line by line
trap '(read -p "[$BASH_SOURCE:$LINENO] $BASH_COMMAND?")' DEBUG
>> you can also customize the prompt with set -x
export PS4='+(${BASH_SOURCE}:${LINENO}) '
set -x
A Customer Acquisition Playbook for Consumer Startups
> For consumer companies, there are only three growth “lanes” that comprise the majority of new customer acquisition:
> 1. Performance marketing (e.g. Facebook and Google ads)
> 2. Virality (e.g. word-of-mouth, referrals, invites)
> 3. Content (e.g. SEO, YouTube)
> There are two additional lanes (sales and partnerships) which we won't cover in this post because they are rarely effective in consumer businesses. And there are other tactics to boost customer acquisition (e.g PR, brand marketing), but the lanes outlined above are the only reliable paths for long-term and sustainable business growth.
Marketing calls those "channels". I don't think they're exclusive categories: a startup's YouTube videos could be supporting a viral marketing campaign, for example; ads aren't the only strategy for (targeted) social media marketing; if the "ask" / desired behavior upon receiving the message is to share the brand, is that "viral"?
What about Super Bowel commercials?
Traditional marketing: press releases, (linked-citation-free) news wires, quasi-paid interviews, "news program" appearances, product placement.
"Growth hacking": https://en.wikipedia.org/wiki/Growth_hacking
Gathering all open and sustainable technology projects
Great list! https://github.com/protontypes/awesome-sustainable-technolog...
"Earth" topic label: https://github.com/topics/earth
Though there are unfortunately zero (0,) results for "sustainability-report[s,ing]", it may be useful to have a heading: https://github.com/topics/sustainability-reporting ... GRI Reporting Standards & XBRL: https://en.wikipedia.org/wiki/Global_Reporting_Initiative , Sustainability Cloud, https://en.wikipedia.org/wiki/Sustainability_reporting
TIL about the "SWEET" (Semantic Web for Earth and Environmental Terminology) Ontologies: https://github.com/ESIPFed/sweet
SunPy: https://docs.sunpy.org/en/stable/generated/gallery/index.htm...
Jupyter Notebooks Gallery
This is not a valid Show HN, so we've taken that out of the title. Please see the rules: https://news.ycombinator.com/showhn.html. Note the word 'lists'.
Lists tend not to make good HN submissions because the only thing one can really discuss about them is the lowest common denominator (or greatest common factor?) of the list elements. It would be better to pick one of the most interesting elements and submit that instead.
HN is itself a list, and a pointer to a pointer to a pointer is too much indirection.
https://hn.algolia.com/?query=denominator%20list%20by:dang&d...
Jupyter/Jupyter > Wiki > "A gallery of interesting Jupyter Notebooks" lists hundreds of notebooks: https://github.com/jupyter/jupyter/wiki/A-gallery-of-interes...
The mybinder.org Grafana dashboard lists the most popular notebook repos in the last hour: https://grafana.mybinder.org/
Jupyter/Nbviewer > FAQ > "How do you choose the notebooks featured on the nbviewer.jupyter.org homepage?" :
> We originally selected notebooks that we found and liked. We are currently soliciting links to refresh the home page using a Google Form. You may also open an issue with your suggestion.
https://nbviewer.jupyter.org/faq#how-do-you-choose-the-noteb...
Google Form: https://docs.google.com/forms/d/e/1FAIpQLSd6AlVvC7KagENypGTc...
Here's the Nbviewer source code. AMP (And https://schema.org/ScholarlyArticle / Book / CreativeWork metadata) could be useful. https://github.com/jupyter/nbviewer
Jupyter notebooks are almost a huge step forward for programming. The instant feedback, saving of partial program state, and visualizations mixed in are amazing and fun to work with. The fact that it's taking place in a browser, you don't have modal editing with Vim, you don't get plugin support (like MyPy), and you don't have intellisense is a huge step backwards.
I'm hoping that someone addresses these issues, I know VSCode is trying, and Scala "Metals" worksheets are another similar concept (though different, it puts variable values in comments next to code as you update the code).
I feel like all of these are a step closer than what the Lighttable/Eve teams were shooting for, but not quite the next generation way of coding that it could be.
I'm a huge fan of rmarkdown since it addresses many of these shortcomings by way of being much simpler than jupyter notebooks. The files are simply text files, and you can write python too if you like, they aren't just for R.
NestedText, a nice alternative to JSON, YAML, TOML
So, this is basically YAML, "but better". I can repeat once more that "easily understood and used by both programmers and non-programmers" is unapologetically stupid concept that can never succeed. So I see how all of this will sound all too familiar to anybody with a little experience, which makes them to automatically dismiss this YAYAML.
But YAML is really quite complicated, and JSON (which shouldn't be used for config files at all) and TOML (which I love and wish it would gain more popularity) aren't exactly alternatives to YAML. So, I would be actually totally ok with "YAML, but better", as a way to deprecate YAML.
Now, it is clear from the start that this cannot deprecate YAML, because it doesn't even have booleans and numbers. But, surprisingly, I can accept this as well: ok, let's just assume that being good at dealing with strings may be enough.
The problem is, it isn't clear at all from the docs, if this is better than YAML at anything. It raises dozens of questions. I'll start with the most basic ones (using [] as a wrapper/delimiter): how do I represent values [ a], [a ], ["a"] and [""] in this file format?
What kind of values are [<whitespace>a] and [a<whitespace>] supposed to be? They look like typical YAML syntax traps to me.
Why should JSON never be used for configuration? It is sufficient for declaratively expressing anything I have encountered. Do we really need references or other stuff from YAML? For configuration this seems unnessecary, provided that the program, which interprets the result of parsing the JSON is well written.
JSON lacks comments and will fail for a missing or extra comma, so it's not great for configuration written by humans.
You can use HJSON which is the json with comments. It's fully compatible with json so easy to introduce into anything that does json. https://hjson.github.io/
JSON5 also supports comments and multiline strings with `\`-escaped newlines: https://json5.org/
Triple-quoted multiline strings like HJSON would be great, too.
From "The description of YAML in the README is inaccurate" https://github.com/KenKundert/nestedtext/issues/10 :
> I will mention something else. The section about the "Norway problem" is not quite accurate. Some YAML loaders do in fact load no as false. These are usually YAML 1.1 loaders. YAML 1.2's default schema is the same as JSON's (only true, false, 'null and numbers are non-strings).
> Any YAML loader is free to use any schema it wants. That is, no loader is required to to load no as false. Good loaders should support multiple schemas and custom schemas. The Norway problem isn't technically a YAML problem but a schema problem.
> imho, YAML's biggest failing to date is not making things like this clear enough to the community.
> Note: PyYAML has a BaseLoader schema that loads all scalar values as strings.
Algorithm discovers how six molecules could evolve into life’s building blocks
It'd be nice if they could do the same for protein synthesis, but that's obviously a much harder problem.
Folding@home https://en.wikipedia.org/wiki/Folding@home :
> Folding@home (FAH or F@h) is a distributed computing project aimed to help scientists develop new therapeutics to a variety of diseases by the means of simulating protein dynamics. This includes the process of protein folding and the movements of proteins, and is reliant on the simulations run on the volunteers' personal computers.
"AlphaFold: Using AI for scientific discovery" (2020) https://deepmind.com/blog/article/AlphaFold-Using-AI-for-sci...
https://www.kdnuggets.com/2019/07/deepmind-protein-folding-u... :
> At last year’s Critical Assessment of protein Structure Prediction competition (CASP13), researchers from DeepMind made headlines by taking the top position in the free modeling category by a considerable margin, essentially doubling the rate of progress in CASP predictions of recent competitions. This is impressive, and a surprising result in the same vein as if a molecular biology lab with no previous involvement in deep learning were to solidly trounce experienced practitioners at modern machine learning benchmarks.
Citations of "Resource-efficient quantum algorithm for protein folding" (2019) https://scholar.google.com/scholar?cites=1037213034434902738...
Protein folding: https://en.wikipedia.org/wiki/Protein_folding
I always wonder how many open research problems could benefit from a working hand from computing. Sometimes I wish research would be as available as an open-source project.
As a computer scientist who worked in a physics lab: a lot. Other sciences could desperately use the help of talented computer scientists. Unfortunately those other sciences are mostly mired in academia nonsense and don't pay well.
"Applied CS"
Computational science: https://en.wikipedia.org/wiki/Computational_science
Computational biology: https://en.wikipedia.org/wiki/Computational_biology
Computational thinking: https://en.wikipedia.org/wiki/Computational_thinking :
> The characteristics that define computational thinking are decomposition, pattern recognition / data representation, generalization/abstraction, and algorithms.
Additional skills useful for STEM fields: system administration / DevOps / DevSecOps, HPC: High Performance Computing (distributed systems, distributed algorithms, performance optimization; rewriting code that is designed to test unknown things with tests and for performance), research a graph of linked resources and reproducibly publish in LaTeX and/or computational notebooks such as Jupyter notebooks, dask-labextension, open source tool development (& sustainable funding) that lasts beyond one grant
Physicists build circuit that generates clean, limitless power from graphene
"Fluctuation-induced current from freestanding graphene" (2020) https://journals.aps.org/pre/abstract/10.1103/PhysRevE.102.0... https://doi.org/10.1103/PhysRevE.102.042101
> In the 1950s, physicist Léon Brillouin published a landmark paper refuting the idea that adding a single diode, a one-way electrical gate, to a circuit is the solution to harvesting energy from Brownian motion. Knowing this, Thibado's group built their circuit with two diodes for converting AC into a direct current (DC). With the diodes in opposition allowing the current to flow both ways, they provide separate paths through the circuit, producing a pulsing DC current that performs work on a load resistor.
> Additionally, they discovered that their design increased the amount of power delivered. "We also found that the on-off, switch-like behavior of the diodes actually amplifies the power delivered, rather than reducing it, as previously thought," said Thibado. "The rate of change in resistance provided by the diodes adds an extra factor to the power."
> The team used a relatively new field of physics to prove the diodes increased the circuit's power. "In proving this power enhancement, we drew from the emergent field of stochastic thermodynamics and extended the nearly century-old, celebrated theory of Nyquist," said coauthor Pradeep Kumar, associate professor of physics and coauthor.
> According to Kumar, the graphene and circuit share a symbiotic relationship. Though the thermal environment is performing work on the load resistor, the graphene and circuit are at the same temperature and heat does not flow between the two.
> That's an important distinction, said Thibado, because a temperature difference between the graphene and circuit, in a circuit producing power, would contradict the second law of thermodynamics. "This means that the second law of thermodynamics is not violated, nor is there any need to argue that 'Maxwell's Demon' is separating hot and cold electrons," Thibado said.
I don't understand. It sounds like they're drawing useful work (I think?) from a system that's already at thermodynamic equilibrium. Doesn't that violate the 2nd law of thermodynamics?
I'm not sure that I understand either. From the abstract (which phys.org failed to link to):
> The system reaches thermal equilibrium and the rates of heat, work, and entropy production tend quickly to zero. However, there is power generated by graphene which is equal to the power dissipated by the load resistor.
Looks like the article is also on ArXiV: https://arxiv.org/abs/2002.09947
https://scholar.google.com/scholar?oi=bibs&hl=en&cluster=103...
Is it really a closed system at equilibrium?
Mozilla shuts project Iodide: Datascience documents in browsers
I did this! I killed it and I didn't mean to.
Ten (10) days ago, I filed an issue in the iodide project: "Compatibility with 'percent' notebook format" https://github.com/iodide-project/iodide/issues/2942
And then six (6) days ago, I added this comment to that issue: https://github.com/iodide-project/iodide/issues/2942#issueco...
And now it's almost dead, and I didn't mean to kill it.
But I also suggested that it would be great if conda-forge had a WASM build target:
- "Consider moving CPython patches upstream" https://github.com/iodide-project/pyodide/issues/635#issueco...
For students, being able to go to a URL and have a notebook interface with the SciPy stack preinstalled without needing to have an organization manage shell accounts and/or e.g. JupyterHub for every student should be worth the necessary budget allocation. Their local machines have plenty of CPU, storage, and memory for all but big data workloads.
Iodide is/was really cool. Pyiodide (much of the SciPy stack compiled to WASM) is also a great idea.
Jyve with latest JupyterLab, nbgrader, and configurable cloud storage could also solve.
Google Colab is an alternative to consider if you need to share Jupyter notebooks: https://colab.research.google.com/notebooks/intro.ipynb.
There are many ways to share reproducible Jupyter notebooks.
Google Colab now supports ipywidgets (js) in notebooks. While you can install additional packages in Colab, additional packages must be installed by each user (e.g. with `! pip install sympy` in an initial input cell) for each new kernel.
repo2docker builds a docker image from software dependency versions specified in e.g. requirements.txt, environment.yml, and/or a postInstall script and then installs a current version of JupyterLab in the container. Zero-to-BinderHub describes how to get BinderHub (which builds and launches containers) running on a hosting provider w/ k8s. awesome-python-in-education/blob/master/README.md#jupyter
Google AI Platform Notebooks is hosted JupyterLab.
awesome-jupyter > Hosted Notebook Solutions lists a number of services: https://github.com/markusschanta/awesome-jupyter#hosted-note...
awesome-python-in-education > Jupyter links to many Jupyter resources like nbgrader and BinderHub but not yet Jyve: https://github.com/quobit/awesome-python-in-education#jupyte...
Ask HN: What are good life skills for people to learn?
My initial thoughts; learn to drive, first aid, a sport, play an instrument, a language, how to manage finances, to speak in front of people.
- "Consumer science (a.k.a. home economics) as a college major" https://news.ycombinator.com/item?id=17894550
In no particular order:
- Food science; Nutrition
- Family planning: https://en.wikipedia.org/wiki/Family_planning
- Personal finance (see the link above for resources)
- How to learn
- How to teach [reading and writing, STEM, respect, compassion]
- Compassion for others' suffering
- How to considerately escape from unhealthy situations
- Coping strategies: https://en.wikipedia.org/wiki/Coping
- Defense mechanisms: https://en.wikipedia.org/wiki/Defence_mechanism
- Prioritization; productivity
- Goal setting; n-year planning; strategic alignment
Life skills: https://en.wikipedia.org/wiki/Life_skills
Khan Academy > Life Skills: https://www.khanacademy.org/college-careers-more
Four Keys Project metrics for DevOps team performance
> […] four key metrics that indicate the performance of a software development team:
> Deployment Frequency - How often an organization successfully releases to production
> Lead Time for Changes - The amount of time it takes a commit to get into production
> Change Failure Rate - The percentage of deployments causing a failure in production
> Time to Restore Service - How long it takes an organization to recover from a failure in production
Ask HN: Resources to encourage teen on becoming computer engineer?
Howdy HN
A teenager I am close with would like to become a computer engineer. Whet resources, books, podcasts, camps, or experiences do you recommend to support this teen's endeavor?
"Ask HN: Something like Khan Academy but full curriculum for grade schoolers?" [through undergrads] https://news.ycombinator.com/item?id=23794001
"Ask HN: How to introduce someone to programming concepts during 12-hour drive?" https://news.ycombinator.com/item?id=15454071
"Ask HN: Any detailed explanation of computer science" https://news.ycombinator.com/item?id=15270458 : topologically-sorted? Information Theory and Constructor Theory are probably at the top:
> A bottom-up (topologically sorted) computer science curriculum (a depth-first traversal of a Thing graph) ontology would be a great teaching resource.
> One could start with e.g. "Outline of Computer Science", add concept dependency edges, and then topologically (and alphabetically or chronologically) sort.
> https://en.wikipedia.org/wiki/Outline_of_computer_science
> There are many potential starting points and traversals toward specialization for such a curriculum graph of schema:Things/skos:Concepts with URIs.
> How to handle classical computation as a "collapsed" subset of quantum computation? Maybe Constructor Theory?
> https://en.wikipedia.org/wiki/Constructor_theory
https://westurner.github.io/hnlog/ ... Ctrl-F "interview", "curriculum"
CadQuery: A Python parametric CAD scripting framework based on OCCT
I wish more people would try this over openscad. I like openscad but the UI and with cadquery you use python! and now I noticed "CadQuery supports Jupyter notebook out of the box"
The jupyter-cadquery extension renders models with three.js via pythreejs in a sidebar with jupyterlab-sidecar: https://github.com/bernhard-42/jupyter-cadquery#b-using-a-do...
https://github.com/bernhard-42/jupyter-cadquery/blob/master/...
Array Programming with NumPy
Looks like there's a new citation for NumPy in town.
"Citing packages in the SciPy ecosystem" lists the existing citations for SciPy, NumPy, scikits, and other -Py things: https://www.scipy.org/citing.html ( source: https://github.com/scipy/scipy.org/blob/master/www/citing.rs... )
A better way to cite requisite software might involve referencing a https://schema.org/SoftwareApplication record in JSON-LD, RDFa, or Microdata; for example: https://news.ycombinator.com/item?id=24489651
But there's as of yet no way to publish JSON-LD, RDFa, or Microdata Linked Data from LaTeX with Computer Modern.
For some reason this struck me as inappropriate for the outlet. It's a nice piece as an introduction to array programming with numpy, but seemed out of place to me.
If, going forward, 5% of all papers that use NumPy to get their results actually cite this paper, it will be one of Nature's most cited papers every year.
There's an interesting trend of what content gets published in peer-reviewed journals vs. blogs/github/etc. I suspect there is an audience segment that strongly values peer reviewed pieces that are equivalent content wise to introductory material in a variety of formats.
I wonder if github should add a "Review" feature to provide a similar content authoring experience.
It would be nice if citing repositories were easier-- either for generating a reference for my own code or acknowledging when I've used someone else's code in my research.
There's tons of math and physics blogs that contain useful results that the author wanted to make available but didn't manage to incorporate into a paper. I wonder if there'd be any interest in a sort of GitHub for proofs? It could even use git, since (assuming consistency) isn't math just a DAG anyways (and therefore isomorphic to a neural net, as are all things).
You can get a free DOI for and archive a tag of a Git repo with FigShare or Zenodo.
If you have repo2docker REES dependency scripts (requirements.txt, environment.yml, postInstall,) in your repo, a BinderHub like https://mybinder.org can build and cache a container image and launch a (free) instance in a k8s cloud.
Journals haven't yet integrated with BinderHub.
Putting the suggested citation and DOI URI/URL in your README and cataloging citations in an e.g. wiki page may increase the crucial frequency of citation.
A Linked Data format for presenting well-formed arguments with #StructuredPremises would help to realize the potential of the web as a graph of resources which may satisfy formal inclusion criteria for #LinkedMetaAnalyses.
The issue is that none of the citation count engines (Google scholar, scopus, Web of Science...) count citations on those DOIs. So for a researcher who needs to somehow demonstrate impact through citation counts, it does not really help unfortunately.
We could reason about sites that index https://schema.org/ScholarlyArticle according to our own and others' observations. Google Scholar, Semantic Scholar, and Meta all index Scholarly Articles: they copy the bibliographic metadata and the abstract for archival and schoarly purposes.
AFAIU, e.g. Zotero and Mendeley do not crawl and index articles or attempt to parse bibliographic citations from the astounding plethora of citation styles [citationstyles, citationstyles_stylerepo] into a citation graph suitable for representative metrics [zenodo_newmetrics].
bitcoin.org/bitcoin.pdf does not have a DOI, does not have an ORCID [orcid], and is not published in any journal but is indexed by e.g. Google Scholar; though there are apparently multiple records referring to a ScholarlyArticle with the same name and author. Something like "Hell's Angels" (1930)? No DOI, no ORCID, no parseable PDF structure: not indexed.
AFAIU, Google Scholar does not yet index ScholarlyArticle (or SoftwareApplication < CreativeWork) bibliographic metadata. GScholar indexes an older set of bibliographic metadata from HTML <meta> tags and also attempts to parse PDFs. [gscholar_inclusion]
Google Scholar is also not (yet?) integrated with Google Dataset Search (which indexes https://schema.org/Dataset metadata).
FigShare DOIs and Zenodo DOIs are DataCite DOIs [figshare_howtocite, zenodo_principles]; which apparently aren't (yet?) all indexed by Google Scholar [rescience_gscholar].
IIUC, all papers uploaded to https://arxiv.org are indexed by Google Scholar. In order for arxiv-vanity.org [arxiv_vanity] to render a mobile-ready, font-resizeable HTML5 version of a paper uploaded to ArXiV, the PostScript source must be uploaded. Arxiv hosts certain categories of ScholarlyArticles.
JOSS (Journal of Open Source Software) has managed to get articles indexed by Google Scholar [rescience_gscholar]. They publish their costs [joss_costs]: $275 Crossref membership, DOIs: $1/paper:
> Assuming a publication rate of 200 papers per year this works out at ~$4.75 per paper
[citationstyles]: https://citationstyles.org
[citationstyles_stylerepo]: https://github.com/citation-style-language/styles
[gscholar_inclusion]: https://scholar.google.com/intl/en/scholar/inclusion.html#in...
[figshare_howtocite]: https://knowledge.figshare.com/articles/item/how-to-share-ci...
[zenodo_principles]: https://about.zenodo.org/principles/
[zenodo_newmetrics]: https://www.frontiersin.org/articles/10.3389/frma.2017.00013...
[rescience_gscholar]: https://github.com/ReScience/ReScience/issues/38
[arxiv_vanity]: https://www.arxiv-vanity.com/
[joss_costs]: https://joss.theoj.org/about#costs
[orcid]: https://en.wikipedia.org/wiki/ORCID
Do you like the browser bookmark manager?
How do you think it compares to services like webcull.com, raindrop.io, or getpocket.com? Have they advanced the field to the point that it's worth switching?
Things I'd add to browser bookmark managers someday:
- Support for (persisting) bookmarks tags. From the post re: the re-launch of del.icio.us: https://news.ycombinator.com/item?id=23985623
> "Allow reading and writing bookmark tags" https://bugzilla.mozilla.org/show_bug.cgi?id=1225916
> Notes re: how this could be standardized with JSON-LD: https://bugzilla.mozilla.org/show_bug.cgi?id=1225916#c116
> The existing Web Experiment for persisting bookmark tags: https://github.com/azappella/webextension-experiment-tags/bl...
- Standard search features like operators: ((term) AND (term2)) OR term3
- Regex search
- (Chrome) show the createdDate and allow (non-destructive) sort by date
- Native sync API for syncing to zero or more bookmarks / personal data storage providers
- Support for integration with extensions that support actual resource metadata like Zotero
- Linked Data support: extract and store bibliographic metadata like Zotero and OpenLink Structured Data Sniffer
What are the current limitations of the WebExtensions Bookmarks API (now supported by Firefox, Chrome, Edge, and hopefully eventually Safari)?: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/Web...
NIST Samate – Source Code Security Analyzers
Additional lists of static analysis, dynamic analysis, SAST, DAST, and other source code analysis tools:
OWAP > Source Code Analysis Tools: https://owasp.org/www-community/Source_Code_Analysis_Tools
https://analysis-tools.dev/ (supports upvotes and downvotes)
analysis-tools-dev/static-analysis: https://github.com/analysis-tools-dev/static-analysis
analysis-tools-dev/dynamic-analysis: https://github.com/analysis-tools-dev/dynamic-analysis
devsecops/awesome-devsecops: https://github.com/devsecops/awesome-devsecops , https://github.com/TaptuIT/awesome-devsecops
kai5263499/awesome-container-security: https://github.com/kai5263499/awesome-container-security
https://en.wikipedia.org/wiki/DevOps#DevSecOps,_Shifting_Sec... :
> DevSecOps is an augmentation of DevOps to allow for security practices to be integrated into the DevOps approach. The traditional centralised security team model must adopt a federated model allowing each delivery team the ability to factor in the correct security controls into their DevOps practices.
awesome-safety-critical: https://awesome-safety-critical.readthedocs.io/en/latest/
A Handwritten Math Parser in 100 lines of Python
Slightly more amusingly, but non-equivalently:
import sys
from functools import reduce
from operator import add, sub, mul, truediv as div
def calc(s, ops=[(add, '+'), (sub, '-'), (mul, '*'), (div, '/')]):
if not ops:
return float(s)
(func, token), *remaining_ops = ops
return reduce(func, (calc(part, remaining_ops) for part in s.split(token)))
if __name__ == '__main__':
print(calc(''.join(sys.argv[1:])))
This is pretty clever. There's a video by Jonathan Blow, creator of Braid and currently making a programming language called Jai, that talks about parsing expressions with order of operations. He gives an simple approach that apparently was discovered recently (though he didn't remember the paper name) that does this in a single pass and hardly any more complicated than what you wrote. You can see it here [1]. He also trashes on using yacc/bison before that and on most academic parsing theory.
Reverse Polish notation (RPN) > Converting from infix notation https://en.wikipedia.org/wiki/Reverse_Polish_notation#Conver... > Shunting-yard algorithm https://en.wikipedia.org/wiki/Shunting-yard_algorithm
Infix notation supports parentheses.
Infix notation: 3 + 4 × (2 − 1)
RPN: 3 4 2 1 − × +
PEP – An open source PDF editor for Mac
I have a dream... that one day people will name their projects with names that don't exist on google yet.
If you search for PEP now you'll find python enhancement proposals, and the "Philippine Entertainment Portal" and the stock code for PepsiCo.
I wish that once people do name their project, they would assign it a 128-bit random number in lower case hex, and include that number on any web page that they would like people searching for their project to find.
That way once I know that say PEP the PDF editor exists and find its 128-bit number (let's say that is 379dd864b16eaca3ce94c15a6bdfcc73), at least I can subsequently toss a +379dd864b16eaca3ce94c15a6bdfcc73 on my searches to effectively let the search engine know I want PEP the PDF editor results rather than PEP the python enhancement results or PEP the entertainment portal results or PEP that refreshing beverage company stock symbol.
"xxd -l 16 -p /dev/urandom" is a handy way to get a 128-bit random hex number. A UUID generator works, too, although they usually include some punctuation you will need to delete and you might have to lower case their output.
> I wish that once people do name their project, they would assign it a 128-bit random number in lower case hex
We already have something similar: URLs.
tzs is proposing URNs rather than textual program names. A URL is unnecessarily specific (though I suppose you could anycast URL resolution)
> RFC 4122 defines a Uniform Resource Name (URN) namespace for UUIDs. A UUID presented as a URN appears as follows:[1]
> > urn:uuid:123e4567-e89b-12d3-a456-426655440000
https://en.wikipedia.org/wiki/Universally_unique_identifier#...
Version 4 UUIDs have 122 random bits (out of 128 bits total).
In Python:
>>> import uuid
>>> _id = uuid.uuid4()
>>> _id.urn
'urn:uuid:4c466878-a81b-4f22-a112-c704655fa4ee'
>>> _id.hex
'4c466878a81b4f22a112c704655fa4ee'
{"@context": "https://schema.org",
"@type": "WebPage",
"about": {
"@type": "SoftwareApplication",
"identifier": "urn:uuid:4c466878-a81b-4f22-a112-c704655fa4ee",
"url": ["", ""],
"name": [
"a schema.org/SoftwareApplication < CreativeWork < Thing",
{"@value": "a rose by any other name",
"@language": "en"}]}}
<body vocab="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" rel="nofollow noopener" target="_blank">https://schema.org/" typeof="WebPage">
<div property="about" typeof="SoftwareApplication">
<meta property="identifier" content="urn:uuid:4c466878-a81b-4f22-a112-c704655fa4ee"/>
<span property="name">a schema.org/SoftwareApplication < CreativeWork < Thing</span>
<span property="name" lang="en">a rose by any other name</span>
</div>
</body>
<div itemtype="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" rel="nofollow noopener" target="_blank">https://schema.org/WebPage" itemscope>
<link itemprop="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" rel="nofollow noopener" target="_blank">http://www.w3.org/ns/rdfa#usesVocabulary" href="https://schema.org/" />
<div itemprop="about" itemtype="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">https://schema.org/SoftwareApplication" itemscope>
<meta itemprop="identifier" content="urn:uuid:4c466878-a81b-4f22-a112-c704655fa4ee" />
<meta itemprop="name" content="a <a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="<a href="http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">http://schema.org/SoftwareApplication" rel="nofollow noopener" target="_blank">schema.org/SoftwareApplication < CreativeWork < Thing"/>
<meta itemprop="name" content="a rose by any other name" lang="en"/>
</div>
</div>The Unix timestamp will begin with 16 this Sunday
Redox: Unix-Like Operating System in Rust
I'm Jeremy Soller, the creator of Redox OS. Let me know if you have questions!
Are there tools to support static analysis and formal methods in Rust yet?
From https://news.ycombinator.com/item?id=21839514 re: awesome-safety-critical https://awesome-safety-critical.readthedocs.io/en/latest/ :
> > Does Rust have a chance in mission-critical software? (currently Ada and proven C niches) https://www.reddit.com/r/rust/comments/5iv5j7/does_rust_have...
FWIU, Sealed Rust is in progress.
And there's also RustPython for the userspace.
Ask HN: How are online communities established?
HN, Reddit, Stack Overflow, etc. are all established communities with users. How do you start a community when you don't have any users?
You should read the book People Powered by Jono Bacon. Has some really good insights and is a 101 course on exactly this.
Seconded. "People Powered: How Communities Can Supercharge Your Business, Brand, and Teams" (2019) https://g.co/kgs/CF5TEk
"The Art of Community: Building the New Age of Participation" (2012) https://g.co/kgs/P2V1kn
"Tribes: We need you to lead us" (2011) https://g.co/kgs/T8jaFS
The 1% 'rule' https://en.wikipedia.org/wiki/1%25_rule_(Internet_culture) :
> In Internet culture, the 1% rule is a rule of thumb pertaining to participation in an internet community, stating that only 1% of the users of a website add content, while the other 99% of the participants only lurk. Variants include the 1–9–90 rule (sometimes 90–9–1 principle or the 89:10:1 ratio),[1] which states that in a collaborative website such as a wiki, 90% of the participants of a community only consume content, 9% of the participants change or update content, and 1% of the participants add content.
... Relevant metrics:
- Marginal cost of service https://en.wikipedia.org/wiki/Marginal_cost
- Customer acquisition cost: https://en.wikipedia.org/wiki/Customer_acquisition_cost
- [Quantifiable and non-quantifiable] Customer Lifetime Value: https://en.wikipedia.org/wiki/Customer_lifetime_value
Last words of the almost-cliche community organizer surrounded by dormant accounts: "Network effects will result in sufficient (grant) funding"
Business model examples that may be useful for building and supporting sustainable communities with clear Missions, Objectives, and Criteria for Success: https://gist.github.com/ndarville/4295324
Python Documentation Using Sphinx
I usually generate new Python projects with a cookiecutter; such as cookiecutter-pypackage. I like the way that cookiecutter-pypackage includes a Makefile which has a `docs` task so that I can call `make docs` to build the sphinx docs in the docs/ directory which include:
- a /docs/readme.rst that includes the /README.rst as the first document in the toctree
- a sensible set of default documents: readme (.. include:: /README.rst), installation, usage, modules (sphinx-autodoc output), contributing, authors, history (.. include:: /HISTORY.rst)
- a sphinx conf.py that sets the docs' version and release attributes to pkgname.__version__; so that the version number only needs to be changed in one place (as long as setup.py or setup.cfg also read the version string from pkgname.__version__)
- a default set of extensions: ['sphinx.ext.autodoc', 'sphinx.ext.viewcode'] that generates API docs and includes '[source]' hyperlinks from the generated API docs to the transcluded syntax-highlighted source code and links back to the API docs from the source code
https://github.com/audreyfeldroy/cookiecutter-pypackage/tree...
There are a few styles of docstrings that Sphinx can parse and include in docs with e.g. sphinx-autodoc:
`:param, :type, :returns, :rtype` docstrings (which OP uses; and which pycontracts can read runtime parameter and return type contracts from https://andreacensi.github.io/contracts/ (though Python 3 annotations are now the preferred style for compile or editing-time typechecks))
Numpydoc docstrings: https://numpydoc.readthedocs.io/en/latest/format.html
Googledoc docstrings: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/
You can use Markdown with Sphinx in at least three ways:
MyST Markdown supports Sphinx and Docutils roles and directives. Jupyter Book builds upon MyST Markdown. With Jupyter Book, you can include Jupyter notebooks (which can include MyST Markdown) in your Sphinx docs. Executable notebooks are a much easier way to include up-to-date code outputs in docs. https://myst-parser.readthedocs.io/en/latest/
Sphinx (& ReadTheDocs) w/ recommonmark: https://docs.readthedocs.io/en/stable/intro/getting-started-...
Nbsphinx predates Jupyter Book and doesn't yet support MyST Markdown, but does support Markdown cells in Jupyter notebooks. Nbsphinx includes a parser for including .ipynb Jupyter notebooks in Sphinx docs. nbsphinx supports raw RST (ReST) cells in Jupyter notebooks and has great docs: https://nbsphinx.readthedocs.io/en/latest/
Nbdev is another approach; though it's not Sphinx:
> nbdev is a library that allows you to fully develop a library in Jupyter Notebooks, putting all your code, tests and documentation in one place.
> [...] Add %nbdev_export flags to the cells that define the functions you want to include in your python modules
https://github.com/fastai/nbdev
A few additional sources of docs for Sphinx and ReStructuredText:
Read The Docs docs > Getting Started with Sphinx > External Resources https://docs.readthedocs.io/en/stable/intro/getting-started-...
CPython Devguide > "Documenting Python" https://devguide.python.org/documenting/
"How to write [Linux] kernel documentation" https://www.kernel.org/doc/html/latest/doc-guide/index.html
awesome-sphinxdoc: https://github.com/yoloseem/awesome-sphinxdoc
... "Ask HN: Recommendations for Books on Writing [for engineers]?" https://news.ycombinator.com/item?id=23945580
Traits of good remote leaders
And the evidence for these considerations is what?
They reference a study: https://link.springer.com/article/10.1007%2Fs10869-020-09698..., titled: "Who Emerges into Virtual Team Leadership Roles? The Role of Achievement and Ascription Antecedents for Leadership Emergence Across the Virtuality Spectrum".
Fortunately the references are free to view.
"Table 4 – Correlation of Development Phases, Coping Stages and Comfort Zone transitions and the Performance Model" in "From Comfort Zone to Performance Management" White (2008) tabularly correlates the Tuckman group development phases (Forming, Storming, Norming, Performing, Adjourning) with the Carnall coping cycle (Denial, Defense, Discarding, Adaptation, Internalization) and Comfort Zone Theory (First Performance Level, Transition Zone, Second Performance Level), and the White-Fairhurst TPR model (Transforming, Performing, Reforming). The ScholarlyArticle also suggests management styles for each stage (Commanding, Cooperative, Motivational, Directive, Collaborative); and suggests that team performance is described by chained power curves of re-progression through these stages.
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C43&q=%E2...
IDK what's different about online teams in regards to performance management?
Show HN: Eiten – open-source tool for portfolio optimization
Is it possible to factor (e.g. GRI) sustainability criteria into the portfolio fitness function? https://news.ycombinator.com/item?id=21922558
My concern is that - like any other portfolio optimization algorithm - blindly optimizing on fundamentals and short term returns will lead to investing in firms who just dump external costs onto people in the present and future; so, screening with sustainability criteria is important to me.
From https://news.ycombinator.com/item?id=19111911 :
> awesome-quant lists a bunch of other tools for algos and superalgos: https://github.com/wilsonfreitas/awesome-quant
This is perfect. Thank you for sharing this.
I might start implementing some of these but would love for someone else to add a few PRs as well. The code is pretty modular especially if we want to add new strategies.
(Sustainable) Index ETFs in the stocks.txt universe would likely be less sensitive to single performers' effects in unbalanced portfolios.
> pyfolio.tears.create_interesting_times_tear_sheet measures algorithmic trading algorithm performance during "stress events" https://github.com/quantopian/pyfolio/blob/03568e0f328783a6a...
Ask HN: Any well funded tech companies tackling big, meaningful problems?
Are there any well funded tech startups / companies tackling major societal problems? Any of these fair game: https://en.wikipedia.org/wiki/List_of_global_issues
----
I don't see or hear of any and want to know if this is just my bias or if there really is a shortage of resources in tech being allocated to solving the worlds most important problems. I'm sure I'm not the only engineer that's looking out for companies like this.
Ran into this previous Ask HN (https://news.ycombinator.com/item?id=24168902) that asked a similar question. However, here I wanna focus on the better funded efforts (not side projects, philanthropy etc).
One example I've heard so far is Tesla. Any others?
You can make an impact by solving important local and global problems by investing your time, career, and savings; by listing and comparing solutions.
As a labor market participant, you can choose to work for places that have an organizational mission that strategically aligns with local, domestic, and international objectives.
https://en.wikipedia.org/wiki/Strategic_alignment ... "Schema.org: Mission, Project, Goal, Objective, Task" https://news.ycombinator.com/item?id=12525141
As an investor, you can choose to invest in organizations that are making the sort of impact you're looking for: you can impact invest.
https://en.wikipedia.org/wiki/Impact_investing
You mentioned "List of global issues"; which didn't yet have a link to the UN Sustainable Development Goals (the #GlobalGoals). I just added this to the linked article:
> As part of the 2030 Agenda for Sustainable Development, the UN Millenium Development Goals (2000-2015) were superseded by the UN Sustainable Development Goals (2016-2030), which are also known as The Global Goals. There are associated Targets and Indicators for each Global Goal.
There are 17 Global Goals.
Sustainability reporting standards can align with the Sustainable Development Goals. For example, the GRI standards are now aligned with the UN Sustainable Development Goals.
https://en.wikipedia.org/wiki/Sustainable_Development_Goals
Investors, fund managers, and potential employees can identify companies which are making an impact by reviewing corporate sustainability and ESG reports.
From https://www.undp.org/content/undp/en/home/sustainable-develo... :
> SDG Target 12.6: "Encourage companies, especially large and transnational companies, to adopt sustainable practices and to integrate sustainability information into their reporting cycle"
From https://news.ycombinator.com/item?id=21302926 :
> > What are some of the corporate sustainability reporting standards?
> > From https://en.wikipedia.org/wiki/Sustainability_reporting#Initi... :
> >> Organizations can improve their sustainability performance by measuring (EthicalQuote (CEQ)), monitoring and reporting on it, helping them have a positive impact on society, the economy, and a sustainable future. The key drivers for the quality of sustainability reports are the guidelines of the Global Reporting Initiative (GRI),[3] (ACCA) award schemes or rankings. The GRI Sustainability Reporting Guidelines enable all organizations worldwide to assess their sustainability performance and disclose the results in a similar way to financial reporting.[4] The largest database of corporate sustainability reports can be found on the website of the United Nations Global Compact initiative.
> >The GRI (Global Reporting Initiative) Standards are now aligned with the UN Sustainable Development Goals (#GlobalGoals). https://en.wikipedia.org/wiki/Global_Reporting_Initiative
> >> In 2017, 63 percent of the largest 100 companies (N100), and 75 percent of the Global Fortune 250 (G250) reported applying the GRI reporting framework.[3]
What are some good ways to search for companies who (1) do sustainability reports, (2) engage in strategic alignment in corporate planning sessions, (3) make sustainability a front-and-center issue in their company's internal and external communications?
What are some examples of companies who have a focus on sustainability and/or who have developed a nonprofit organization for philanthropic missions which are sometimes best accounted for as a distinct organization or a business unit (which can accept and offer receipts for donations as a non-profit)?
How can an employee drive change in a small or a large company? Identify opportunities to deliver value and goodwill. Read through the Global Goals, Targets, and Indicators; and get into the habit of writing down problems and solutions.
3 pillars of [Corporate] Sustainability: (Environment (Society (Economy))). https://en.wikipedia.org/wiki/Sustainability#Three_dimension...
"Launch HN: Charityvest (YC S20) – Employee charitable funds and gift matching" https://news.ycombinator.com/item?id=23907902 :
> We created a modern, simple, and affordable way for companies to include charitable giving in their suite of employee benefits.
> We give employees their own tax-deductible charitable giving fund, like an “HSA for Charity.” They can make contributions into their fund and, from their fund, support any of the 1.4M charities in the US, all on one tax receipt.
> Using the funds, we enable companies to operate gift matching programs that run on autopilot. Each donation to a charity from an employee is matched automatically by the company in our system.
> A company can set up a matching gift program and launch giving funds to employees in about 10 minutes of work.
"Salesforce Sustainability Cloud Becomes Generally Available" https://news.ycombinator.com/item?id=22068522 :
> Are there similar services for Sustainability Reporting and accountability?
Column Names as Contracts
I really like the idea of being more thoughtful about naming columns and being more explicit about the “type” of data contained in them.
Is this idea already known among data modelers or data engineers?
I’d love to read any other references, if available.
I am a data architect in my day job. Within the realm of data management, I'd say "metadata management" [1] is the general category this fits within.
I would say, yes this idea is known/very common, as data architecture is as much about the descriptive language we use as anything. I mean, "business glossaries", taxonomy, even just naming conventions [2] in coding, these are all related.
If you build enough databases/tables or even code yourself, you inevitably come across the "how to name things" problem [3]. If all you have to sort on for the known meaning of a thing (column, table, file, etc.) is a single string value, then encoding meaning into it is quite common. This way, a sort creates a kind of "grouping". Many database vendors follow standard naming conventions - such as Oracle, for example [4]. It is considered a best practice when designing/building the metadata for a large system, to establish a naming convention. Among other things, it makes finding things easier, as well as all the potential for automation.
You get all kinds of variations on this, such as, should the "ID_" come as a prefix or a suffix (i.e. "_ID"). One's initial thought is to use it as a prefix so all the related types group together, but then that becomes much more difficult if you want to sort items by their functional area (e.g. DRIVER_ID, DRIVER_IND, etc.).
One other place you see something similar is in "smart numbers" which is an eternal argument - should I use a "dumb identifier" (GUID, integer) or a "smart one" (one encoding additional meaning) [5].
I mean, basically, any time you can encode information in the meta-data of data, I think you can then operate on it by following "convention over configuration" (as mentioned elsewhere in the discussion comments).
The only problem I see is that such conventions can, at times be limiting - depending on the length of your metadata columns, and the variability you are trying to capture - which is why I believe, generally, metadata is often better separated and linked to the data it describes - this decoupling allows for much more descriptive metadata than one could encode in simple a single string value. Certainly, you can get a long way with an approach like this, but I suspect you would run into 80/20 rule limitations.
Using naming in this way is a form of tight coupling, which could be seen as an anti-pattern in terms of meta-data flexibility, in some cases.
[1] https://en.wikipedia.org/wiki/Metadata_management
[2] https://en.wikipedia.org/wiki/Naming_convention_(programming...
[3] https://martinfowler.com/bliki/TwoHardThings.html
[4] https://oracle-base.com/articles/misc/naming-conventions
In terms of database normalization, delimiting multiple fields within a column name field violates the "atomic columns" requirement of the first though sixth normal forms (1NF - 6NF)
https://en.wikipedia.org/wiki/Database_normalization
Are there standards for storing columnar metadata (that is, metadata about the columns; or column-level metadata)?
In terms of columns, SQL has (implicit ordinal, name, type) and then primary key, index, and [foreign key] constraints.
RDFS (RDF Schema) is an open W3C linked data standard. An rdf:Property may have a rdfs:domain and a rdfs:range; where the possible datatypes are listed as instances of rdfs:range. Primitive datatypes are often drawn from XSD (XML Schema Definition), or https://schema.org/ . An rdfs:Class instance may be within the rdfs:domain and/or the rdfs:range of an rdf:Property.
RDFS is generally not sufficient for data validation; there are a number of standards which build upon RDFS: W3C SHACL (Shapes and Constraint Language), W3C CSVW (CSV on the Web).
There is some existing work on merging JSON Schema and SHACL.
CSVW builds upon the W3C "Model for Tabular Data and Metadata on the Web"; which supports arbitrary "annotations" on columns. CSVW can be represented as any RDF representation: Turtle/Trig/M3, RDF/XML, JSON-LD.
https://www.w3.org/TR/tabular-data-primer/
https://www.w3.org/TR/tabular-data-model/ :
> an annotated tabular data model: a model for tables that are annotated with metadata. Annotations provide information about the cells, rows, columns, tables, and groups of tables […]
...
From https://twitter.com/westurner/status/901992073846456321 :
> "7 metadata header rows (column label, property URI path, DataType, unit, accuracy, precision, significant figures)" https://wrdrd.github.io/docs/consulting/linkedreproducibilit...
...
From https://twitter.com/westurner/status/1295774405923147778 :
> Relevant: https://discuss.ossdata.org/ topics: "Linked Data formats, tools, challenges, opportunities; CSVW, https://schema.org/Dataset , https://schema.org/ScholarlyArticle " https://discuss.ossdata.org/t/linked-data-formats-tools-chal...
> "A dataframe protocol for the PyData ecosystem" https://discuss.ossdata.org/t/a-dataframe-protocol-for-the-p...
> A .meta protocol should implement the W3C Tabular Data Model: [...]
...
The various methods of doing CSV2RDF and R2RML (SQL / RDB to RDF Mapping) each have a way to specify additional metadata annotations. None stuff data into a column name (which I'm also guilty of doing with e.g. "columnspecs" in a small line-parsing utility called pyline that can cast columns to Python types and output JSON lines).
...
Even JSON5 is insufficient when it comes to representing e.g. complex fractions: there must be a tbox (schema) in order to read the data out of the abox (assertions; e.g. JSON). JSON-LD is sufficient for representation; and there are also specs like RDFS, SHACL, and CSVW.
Abox: https://en.wikipedia.org/wiki/Abox
I see the line of thinking you're going down. There are ISO standards for data types, in a sense I could see why one would seek a standard language for defining the metadata/specification of a type as data. Have to really think about that some more.. in a way a regex could be seen as a compact form of expressing the capability of a column in terms of value ranges or domains, but to define the meaning of the data, not so much.
Your interpretation of the atomic columns requirement is a little different than my understanding. That requirement of normalization only applies to the "cells" of columnar data, it says nothing about encoding meaning into column names, which are themselves simply descriptive metadata.
I mean, for sure you wouldn't want to encode many values/meanings into a column name (some systems have length restrictions that would make that impossible, I'm not sure it makes sense anyway), but just pointing out that technically the spec does not make that illegal. Certainly, adding minor annotations within the name of a column separated by a supported delimiter does not, in my opinion, violate normalization rules at all. I mean things like "ID_" or similar.
Have you looked at INFORMATION_SCHEMA in SQL databases? [1] You mentioned SQL metadata and constraints, that is as close to a standard feature for querying that information there is, some databases do it using similar but non-standard ways (Oracle for example).
Also, not standard but, many relational databases support extended properties or Metadata for objects (tables, views, columns, etc.) - you can often come up with your own scheme although rarely do I see people utilize these features. [2] [3]
At some point it feels like we are more talking about type definitions and annotations, applied to data columns.
Maybe like, BNF [4] for purely data table columns (which are essentially types)?
[1] https://en.wikipedia.org/wiki/Information_schema
[2] http://www.postgresql.org/docs/current/static/sql-comment.ht...
[3] https://docs.microsoft.com/en-us/sql/relational-databases/sy...
Graph Representations for Higher-Order Logic and Theorem Proving (2019)
ONNX (and maybe RIF) are worth mentioning.
ONNX: https://onnx.ai/ :
> ONNX is an open format built to represent machine learning models. ONNX defines a common set of operators - the building blocks of machine learning and deep learning models - and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers
RIF (~FOL): https://en.wikipedia.org/wiki/Rule_Interchange_Format
Datalog (not Turing-complete): https://en.wikipedia.org/wiki/Datalog
HOList Benchmark: https://sites.google.com/view/holist/home
"HOList: An Environment for Machine Learning of Higher-Order Theorem Proving" (2019) https://arxiv.org/abs/1904.03241
> Abstract: We present an environment, benchmark, and deep learning driven automated theorem prover for higher-order logic. Higher-order interactive theorem provers enable the formalization of arbitrary mathematical theories and thereby present an interesting, open-ended challenge for deep learning. We provide an open-source framework based on the HOL Light theorem prover that can be used as a reinforcement learning environment. HOL Light comes with a broad coverage of basic mathematical theorems on calculus and the formal proof of the Kepler conjecture, from which we derive a challenging benchmark for automated reasoning. We also present a deep reinforcement learning driven automated theorem prover, DeepHOL, with strong initial results on this benchmark.
I wonder why they don't mention any work based on transformer architectures? The recent work on solving differential equations based on expressions in reverse polish notation seemed like a reasonable idea to apply to theorem proving as well.
Really cool work though!
A transformer is unable to really represent logic, let alone higher order logic and theorem proving.
A transformer is a universal function approximator. The question is whether it can do so reasonably efficiently. Trained on natural language linear sequences, I’m with you. Trained on abstract logical graph representations? I don’t think that question’s answered yet, unless I’m missing something.
How do transformers handle with truth tables, logical connectives, and propositional logic / rules of inference, and first-order logic?
Truth table: https://en.wikipedia.org/wiki/Truth_table
Logical connective: https://en.wikipedia.org/wiki/Logical_connective
Propositional logic: https://en.wikipedia.org/wiki/Propositional_calculus
Rules of inference: https://en.wikipedia.org/wiki/Rule_of_inference
DL: Description logic: https://en.wikipedia.org/wiki/Description_logic (... The OWL 2 profiles (EL, QR, RL; DL, Full) have established decideability and complexity: https://www.w3.org/TR/owl2-profiles/ )
FOL: First-order logic: https://en.wikipedia.org/wiki/First-order_logic
HOL: Higher-order logic: https://en.wikipedia.org/wiki/Higher-order_logic
In terms of regurgitating without critical reasoning?
Critical reasoning: https://en.wikipedia.org/wiki/Critical_thinking
Think about a program that can write code but not execute it. It’s not hard to get a transformer to learn to write python code like merge sort or simple arithmetic code, even though a transformer can’t reasonably learn to sort, nor can it learn simple arithmetic. It’s an important disambiguation. In one view it appears it can’t (learn to sort) and in another it demonstrably can (learn to code up a sort function). It can learn to usefully manipulate the language without needing the capacity to execute the language. They probably can’t learn to “execute” what you’re talking about (execute being a loose analogy), but I’d say the jury’s out on whether they can learn to usefully manipulate it.
Transformer is a function from seq of symbols to seq of symbols. For example, truth table is exactly similar kind of function from variables to [True, False] alphabet.
Transformer can represent some very complex logical operations, and per this article is turing complete: https://arxiv.org/abs/1901.03429, meaning any computable function, including theorem prover can be represented as transformer.
Another question is if it is feasible/viable/rational to build transformer for this? My intuition says: no.
Show HN: Linux sysadmin course, eight years on
Almost eight years ago I launched an online “Linux sysadmin course for newbies” here at HN.
It was a side-project that went well, but never generated enough money to allow me to fully commit to leaving the Day Job. After surviving the Big C, and getting made redundant I thought I might improve and relaunch it commercially – but my doctors are a pessimistic bunch, so it looked like I didn’t have the time.
Instead, I rejigged/relaunched it via a Reddit forum this February as free and open - and have now gathered a team of helpers to ensure that it keeps going each month even after I can’t be involved any longer.
It’s a month-long course which restarts each month, so “Day 1” of September is this coming Monday.
It would be great if you could pass the word on to anyone you know who may be the target market of those who: “...aspire to get Linux-related jobs in industry - junior Linux sysadmin, devops-related work and similar”.
[0] http://www.linuxupskillchallenge.org/
[1] https://www.reddit.com/r/linuxupskillchallenge/
[2] http://snori74.blogspot.com/2020/04/health-status.html
There are a number of resources that may be useful for your curriculum for this project listed in "Is there a program like codeacademy but for learning sysadmin?" https://news.ycombinator.com/item?id=19469266 :
> [ http://www.opsschool.org/ , https://github.com/kahun/awesome-sysadmin/blob/master/README... , https://github.com/stack72/ops-books , https://landing.google.com/sre/books/ , https://response.pagerduty.com/ (Incident Response training)]
To that I'd add that K3D (based on K3S, which is now a CNCF project) runs Kubernetes (k8s) in Docker containers. https://github.com/rancher/k3d
For zero-downtime (HA: High availability) deployments, "Zero-Downtime Deployments To a Docker Swarm Cluster" describes Rolling Updates and Blue-Green Deployments; with illustrations: https://github.com/vfarcic/vfarcic.github.io/blob/master/doc...
For git-push style deployment with more of a least privileges approach (which also has more moving parts) you could take a look at: https://github.com/dokku/dokku-scheduler-kubernetes#function...
And also reference ansible molecule and testinfra for writing sysadmin tests and the molecule vagrant driver for testing docker configurations. https://www.jeffgeerling.com/blog/2018/testing-your-ansible-...
https://molecule.readthedocs.io/en/latest/
https://testinfra.readthedocs.io/en/latest/ :
> With Testinfra you can write unit tests in Python to test actual state of your servers configured by management tools like Salt, Ansible, Puppet, Chef and so on.
> Testinfra aims to be a Serverspec equivalent in python and is written as a plugin to the powerful Pytest test engine.
I wasn't able to find a syllabus or a list of all of the daily posts? Are you focusing on DevOps and/or DevSecOps skills?
EDIT: The lessons are Markdown files in a Git repo: https://github.com/snori74/linuxupskillchallenge
Links to each lesson, the title and/or subjects of the lesson, and the associated reddit posts might be useful in a Table of Contents in the README.md.
Thanks, but most of that would be way over the top for my "newbies".
However, You must be the third or fourth person today to suggest that I add a TOC - so that is something I think I'll need to look at!
Maybe most useful as resources for further study.
Looks like Day 20 covers shell scripting. A few things worth mentioning:
You can write tests for shell scripts and write TAP (Test Anything Protocol) -formatted output: https://testanything.org/producers.html#shell
Quoting in shell scripts is something to be really careful about:
> This and this do different things:
# prints a newline
echo $(echo "-e a\nb")
# prints "-e a\nb"
echo "$(echo "-e a\nb")"
LearnXinYminutes has a good bash reference: https://learnxinyminutes.com/docs/bash/
And an okay Ansible reference, which (like Ops School) we should contribute to: https://learnxinyminutes.com/docs/ansible/
Why do so many pros avoid maintaining shell scripts and writing one-off commands that they'll never remember to run again later?
...
It may be helpful to format these as Jupyter notebooks with input and output cells.
- Ctrl-Shift-Minus splits a cell at the cursor
- M and Y toggle a cell between Markdown and code
If you don't want to prefix every code cell line with a '!' so that the ipykernel Jupyter python kernel (the default kernel) executes the line with $SHELL, you can instead install and select bash_kernel; though users attempting to run the notebooks interactively would then need to also have bash_kernel installed: https://github.com/takluyver/bash_kernel
You can save a notebook .ipynb to any of a number of Markdown and non-Markdown formats https://jupytext.readthedocs.io/en/latest/formats.html#markd... ; unfortunately jupytext only auto-saves to md without output cell content for now: https://github.com/mwouts/jupytext/issues/220
You can make reveal.js slides (that do include outputs) from a notebook: https://gist.github.com/mwouts/04a6dfa571bda5cc59fa1429d1309...
With nbconvert, you can manually save an .ipynb Jupyter notebook as Markdown which includes the cell outputs w/ File > "Download as / Export Notebook as" > "Export notebook to Markdown" or with the CLI: https://nbconvert.readthedocs.io/en/latest/usage.html#conver...
jupyter convert --to markdown
jupyter convert --help
jupyter-book build mybook/
From https://westurner.github.io/tools/#bash :
type bash
bash --help
help help
help type
apropos bash
info bash
man bash
man man
info info
info bash -n "Bash Startup Files"
...
Re: dotfiles, losing commands that should've been logged to HISTFILE when running multiple bash sessions and why I wrote usrlog.sh: https://westurner.github.io/hnlog/#comment-20671184 (Ctrl-F for: "dotfiles", "usrlog.sh", "inputrc")
https://github.com/webpro/awesome-dotfiles
...
awesome-sysadmin > resources: https://github.com/kahun/awesome-sysadmin#resources
Software supply chain security
Estimates of prevalence do assume detection. How would we detect that a dependency that was installed a few deployments and reboots ago was compromised?
How does the classic infosec triad (Confidentiality, Integrity, Availability) apply to software supply chain security?
Confidentiality: Presumably we're talking about open source projects; which aren't confidential. Projects may request responsible disclosure in an e.g. security.txt; and vuln reports may be confidential for at least a little while.
Integrity: Secure transport protocols, checksums, and cryptographic code signing are ways to mitigate data integrity risks. GitHub supports SSH, 2FA, and GPG keys. Can all keys in the package signature keyring be used to sign any package? Can we verify a public key over a different channel? When we specify exact versions of software dependencies, can we also record package hashes which the package installer(s) will verify?
Availability: What are the internal and external data, network, and service dependencies for the development and deployment DevSecOps workflows? Can we deploy from local package mirrors? Who is responsible for securing and updating local package mirrors? Are these service dependencies all HA? Does everything in this system also depend upon the load balancer? Does our container registry support e.g. Docker Notary (TUF)? How should we mirror TUF package repos?
See also: "Guidance for [[transparent] proxy cache] partial mirrors?" https://github.com/theupdateframework/specification/issues/1...
A toolset that answers some of your questions is grafeas- a metadata store at https://github.com/grafeas/grafeas- and kritis, a policy engine at https://github.com/grafeas/kritis.
Cheers.
Thanks for the links. Do you know how this toolset helps to mitigate/prevent what is called in the GitHub blogpost "Supply chain compromises". Quickly checked around and couldn't find anything that applies to the dependencies of applications/binaries before they land into the target runtime (i.e k8s).
Have you seen these preso slides
https://www.slideshare.net/mobile/aysylu/q-con-sp-software-s....
They walk through one of the workflows (end state is deploying to k8s).
Grafeas is a metadata store, Kritis is a policy engine that plugs into k8s as an admission controller- blessing the "admission" (running) of an image in a namespace.
There are existing tools for each language/runtime that produce known vuln lists for individual artifacts in the language ecosystem. These you feed into Grafeas. And you have your CI pipeline providing manifests for each of your built images that contain all upstream dependencies (these produced from each app's build tool). Then at deploy time, Kritis checks the manifest on the image, and for each artifact in the image, checks for vulns and determines whether the vuln should keep the image from being deployed.
Hope that helps. There are many other workflows but that one is the most direct.
Cheers.
OUTSTANDING comment; excellent questions. Bookmarked. Thanks for this concise high-level infosec punchlist.
This Sir is senior.
Mind Emulation Foundation
"While we are far from understanding how the mind works, most philosophers and scientists agree that your mind is an emergent property of your body. In particular, your body’s connectome. Your connectome is the comprehensive network of neural connections in your brain and nervous system. Today your connectome is biological. "
This is a pretty speculative thesis. It's not at all clear that everything relevant to the mind is found in the connections rather than the particular biochemical processes of the brain. It's a very reductionist view that drastically underestimates the biological complexity of even individual cells. There's a good book, Wetware: A Computer in Every Living Cell, by Dennis Bray going into detail on how much functionality and physical processes are at work even in the most simplest cells that is routinely ignored by these analogies of the brain to a digital computer.
There is this extreme, and I would argue unscientific bias towards treating the mind as something that's recreatable in a digital system probably because it enables this science-fiction speculation and dreams of immortality of people living in the cloud.
Indeed. We humans largely create devices that function either through calculation or through physical reaction, relying on the underlying rules of the universe to "do the math" of, say, launching a cannonball and having it follow a consistent arc. The brain combines both at almost every level. It may be fundamentally impossible to emulate a human personality equal to a real one without a physics simulation of a human brain and its chemistry.
A dragonfly brain takes the input from thirty thousand visual receptor cells and uses it to track prey movement using only sixteen neurons. Could we do the same using an equal volume of transistors?
No one is saying a neuron is a one to one equivalent with a transistor. That behavior does seem like it's possible to emulate with many transistors, however.
Was just talking about quantum cognition and memristors (in context to GIT) a few days ago: https://news.ycombinator.com/item?id=24317768
Quantum cognition: https://en.wikipedia.org/wiki/Quantum_cognition
Memristor: https://en.wikipedia.org/wiki/Memristor
It may yet be possible to sufficiently functionally emulate the mind with (orders of magnitude more) transistors. Though, is it necessary to emulate e.g. autonomic functions? Do we consider the immune system to be part of the mind (and gut)?
Perhaps there's something like an amplituhedron - or some happenstance correspondence - that will enable more efficient simulation of quantum systems on classical silicon pending orders of magnitude increases in coherence and also error rate in whichever computation medium.
For abstract formalisms (which do incorporate transistors as a computation medium sufficient for certain tasks), is there a more comprehensive set than Constructor Theory?
Constructor theory: https://en.wikipedia.org/wiki/Constructor_theory
Amplituhedron: https://en.wikipedia.org/wiki/Amplituhedron
What is the universe using our brains to compute? Is abstract reasoning even necessary for this job?
Something worth emulating: Critical reasoning. https://en.wikipedia.org/wiki/Critical_reasoning
13 Beautiful Tools to Enhance Online Teaching and Learning Skills
"Options for giving math talks and lectures online" https://news.ycombinator.com/item?id=22541754 also lists a number of resources for teaching math online.
How close are computers to automating mathematical reasoning?
It's irresponsible to not mention that a series of results from the 1930s through the 1950s proved that generalized automated proof search is impossible, in the sense that we cannot just ask a computer to find a proof or refutation of Goldbach's Conjecture or other "Goldbach-type" statement and have it do any better than a person. In that sense, no, computers will never fully automate mathematical reasoning, because mathematical reasoning cannot be fully automated.
What we can automate, and have successfully automated many times over, is proof checking. In that sense, yes, computers have already fully automated mathematical reasoning, because checking proofs is fully formalized and mechanized.
I think that what people really ought to be asking for is the ease with which we find new results and communicate them to others. In that sense, computer-aided proofs can be hard to read and so there is much work to be done in making them easier to communicate to humans. Similarly, there is interesting work being done on how to make computer-generated proofs which use extremely-high-level axiom schemata to generate human-like handwaving abstractions.
I'm still not sure how I feel about the ATP/ITP terminology used here. ATPs are either of the weak sort that crunch through SAT, graphs, and other complete-but-hard problems, or the strong sort which are impossible. Meanwhile, folks have drifted from "interactive" to "assistant", and talk of "proof assistants" as tools which, like a human, can write down and look at sections of a proof in isolation, but cannot summon complete arbitrary proofs from its own mind.
Edit: One edit will be quicker than two replies and I tend to be "posting too fast". The main point is captured well by [0], but they immediately link to [1], the central result, which links to [2], an important corollary. At this point, I'm just going to quote WP:
> The first [Gödel] incompleteness theorem states that no consistent system of axioms whose theorems can be listed by an effective procedure (i.e., an algorithm) is capable of proving all truths about the arithmetic of natural numbers. For any such consistent formal system, there will always be statements about natural numbers that are true, but that are unprovable within the system.
> The second [Gödel] incompleteness theorem, an extension of the first, shows that the system cannot demonstrate its own consistency.
> Informally, [Tarski's Undefinability] theorem states that arithmetical truth cannot be defined in arithmetic.
I recognize that these statements may seem surprising, but they are provable and you should convince yourself of them. I recently reviewed [3] and found it to be a very precise and complete introduction to all of the relevant ideas; there's also GEB if you want something more fun.
[0] https://en.wikipedia.org/wiki/Automated_theorem_proving#Deci...
[1] https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...
[2] https://en.wikipedia.org/wiki/Tarski%27s_undefinability_theo...
[3] https://www.logicmatters.net/resources/pdfs/godelbook/GodelB...
As someone unfamiliar with the results you describe, why can’t a machine do better than a human? A smart human is better at writing proofs than a dumb one. If we had a highly advanced general artificial intelligence, why couldn’t it generate better results than the smart humans?
Or is automated proof search impossible for humans as well?
Arguably, humans require more energy per operation. So, presumably such an argument hinges upon what types of operations are performed in conducting automated proof search?
What would "automated proof search" even mean in the context of being performed by a human being?
The context here is that certain problems cannot be solved by an algorithm, which doesn't necessarily translate into "cannot be performed by a machine".
It only means that there cannot be any Turing machine capable of solving them, no matter what "operations" it represents.
The task (in terms of constructor theory) is: Find the functions that sufficiently approximate the observations and record their reproducible derivations.
Either the (unreferenced) study was actually arguing that "automated proof search" can't be done at all, or that human neural computation is categorically non-algorothmic.
Grid search of all combinations of bits that correspond to [symbolic] classical or quantum models.
Or better: evolutionary algorithms and/or neural nets.
Roger Penrose argues that human neural computation is indeed non-algorithmic in nature (see his book "The Emperor's New Mind"; 1989) and speculates that quantum processes are involved.
I don't quite understand the role of neural nets in that context, though. Those just classical computations in the end and should be bound by the same limits that every other algorithms are, shouldn't they?
That human cognition is quantum in nature - that e.g. entanglement is necessary - may be unfalsifiable.
Neuromorphic engineering has expanded since the 1980s. https://en.wikipedia.org/wiki/Neuromorphic_engineering
Quantum computing is the best known method for simulating chemical reactions and thereby possibly also neurochemical reactions. But, Is quantum computing necessary to functionally emulate human cognition?
It may be that a different computation medium can accomplish the same tasks without emulating all of the complexity of the brain.
If the brain is only classical and some people are using their brains to perform quantum computations, there may be something there.
Quantum cognition: https://en.wikipedia.org/wiki/Quantum_cognition
Quantum memristors are still elusive.
From "Quantum Memristors in Frequency-Entangled Optical Fields" (2020) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7079656/ :
> Apart from the advantages of using these devices for computation [12] (such as energy efficiency [13], compared to transistor-based computers), memristors can be also used in machine learning schemes [14,15]. The relevance of the memristor lies in its ubiquitous presence in models which describe natural processes, especially those involving biological systems. For example, memristors inherently describe voltage-dependent ion-channel conductances in the axon membrane in neurons, present in the Hodgkin–Huxley model [16,17].
> Due to the inherent linearity of quantum mechanics, it is not straightforward to describe a dissipative non-linear memory element, such as the memristor, in the quantum realm, since nonlinearities usually lead to the violation of fundamental quantum principles, such as no-cloning theorem. Nonetheless, the challenge was already constructively addressed in Ref. [18]. This consists of a harmonic oscillator coupled to a dissipative environment, where the coupling is changed based on the results of a weak measurement scheme with classical feedback. As a result of the development of quantum platforms in recent years, and their improvement in controllability and scalability, different constructions of a quantum memristor in such platforms have been presented. There is a proposal for implementing it in superconducting circuits [7], exploiting memory effects that naturally arise in Josephson junctions. The second proposal is based on integrated photonics [19]: a Mach–Zehnder interferometer can behave as a beam splitter with a tunable reflectivity by introducing a phase in one of the beams, which can be manipulated to study the system as a quantum memristor subject to different quantum state inputs.
Quantum harmonic oscillators have also found application in modeling financial markets. Quantum harmonic oscillator: https://en.wikipedia.org/wiki/Quantum_harmonic_oscillator
New framework for natural capital approach to transform policy decisions
Natural capital: https://en.wikipedia.org/wiki/Natural_capital
> Natural capital is the world's stock of natural resources, which includes geology, soils, air, water and all living organisms. Some natural capital assets provide people with free goods and services, often called ecosystem services. Two of these (clean water and fertile soil) underpin our economy and society, and thus make human life possible.
Natural capital accounting: https://en.wikipedia.org/wiki/Natural_capital_accounting
> Natural capital accounting is the process of calculating the total stocks and flows of natural resources and services in a given ecosystem or region.[1] Accounting for such goods may occur in physical or monetary terms. This process can subsequently inform government, corporate and consumer decision making as each relates to the use or consumption of natural resources and land, and sustainable behaviour.
Opportunity cost: https://en.wikipedia.org/wiki/Opportunity_cost
> When an option is chosen from alternatives, the opportunity cost is the "cost" incurred by not enjoying the benefit associated with the best alternative choice.[1] The New Oxford American Dictionary defines it as "the loss of potential gain from other alternatives when one alternative is chosen."[2] In simple terms, opportunity cost is the benefit not received as a result of not selecting the next best option. Opportunity cost is a key concept in economics, and has been described as expressing "the basic relationship between scarcity and choice". [3] The notion of opportunity cost plays a crucial part in attempts to ensure that scarce resources are used efficiently.[4] Opportunity costs are not restricted to monetary or financial costs: the real cost of output forgone, lost time, pleasure or any other benefit that provides utility should also be considered an opportunity cost. The opportunity cost of a product or service is the revenue that could be earned by its alternative use.
How do we value essential dependencies in terms of future opportunity costs?
In terms of just mental health?
"National parks a boost to mental health worth trillions: study" https://phys.org/news/2019-11-national-boost-mental-health-w...
> Visits to national parks around the world may result in improved mental health valued at about $US6 trillion (5.4 trillion euros), according to a team of ecologists, psychologists and economists
> Professor Bateman's decision-making framework focuses on the links between the environment and economy and has three components: efficiency, assessing which option generates the greatest benefit; sustainability, the effects of each option on natural capital stocks; and equity, regarding who receives the benefits of a decision and when.
Ian J. Bateman et al. "The natural capital framework for sustainably efficient and equitable decision making", Nature Sustainability (2020). DOI: 10.1038/s41893-020-0552-3 https://www.nature.com/articles/s41893-020-0552-3
Challenge to scientists: does your ten-year-old code still run?
"Ten Simple Rules for Reproducible Computational Research" http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fj... :
> Rule 1: For Every Result, Keep Track of How It Was Produced
> Rule 2: Avoid Manual Data Manipulation Steps
> Rule 3: Archive the Exact Versions of All External Programs Used
> Rule 4: Version Control All Custom Scripts
> Rule 5: Record All Intermediate Results, When Possible in Standardized Formats
> Rule 6: For Analyses That Include Randomness, Note Underlying Random Seeds
> Rule 7: Always Store Raw Data behind Plots
> Rule 8: Generate Hierarchical Analysis Output, Allowing Layers of Increasing Detail to Be Inspected
> Rule 9: Connect Textual Statements to Underlying Results
> Rule 10: Provide Public Access to Scripts, Runs, and Results
... You can get a free DOI for and archive a tag of a Git repo with FigShare or Zenodo.
... re: [Conda and] Docker container images https://news.ycombinator.com/item?id=24226604 :
> - repo2docker (and thus BinderHub) can build an up-to-date container from requirements.txt, environment.yml, install.R, postBuild and any of the other dependency specification formats supported by REES: Reproducible Execution Environment Standard; which may be helpful as Docker Hub images will soon be deleted if they're not retrieved at least once every 6 months (possibly with a GitHub Actions cron task)
BinderHub builds a container with the specified versions of software and installs a current version of Jupyter Notebook with repo2docker, and then launches an instance of that container in a cloud.
“Ten Simple Rules for Creating a Good Data Management Plan” http://journals.plos.org/ploscompbiol/article?id=10.1371/jou... :
> Rule 6: Present a Sound Data Storage and Preservation Strategy
> Rule 8: Describe How the Data Will Be Disseminated
... DVC: https://github.com/iterative/dvc
> Data Version Control or DVC is an open-source tool for data science and machine learning projects. Key features:
> - Simple command line Git-like experience. Does not require installing and maintaining any databases. Does not depend on any proprietary online services. Management and versioning of datasets and machine learning models. Data is saved in S3, Google cloud, Azure, Alibaba cloud, SSH server, HDFS, or even local HDD RAID.
> - Makes projects reproducible and shareable; helping to answer questions about how a model was built.
There are a number of great solutions for storing and sharing datasets.
... "#LinkedReproducibility"
Open textual formats for data and open source application and system software (more precisely, FLOSS), are just as important.
Imagine that x86 - and with it, the PC platform - gets replaced by ARM within a decade. For binary software, this would be a kind of geological extinction event.
The likelihood of there being a [security] bug discovered in a given software project over any significant period of time is near 100%.
It's definitely a good idea to archive source and binaries and later confirm that the output hasn't changed with and without upgrading the kernel, build userspace, execution userspace, and PUT/SUT Package/Software Under Test.
- Specify which versions of which constituent software libraries are utilized. (And hope that a package repository continues to serve those versions of those packages indefinitely). Examples: Software dependency specification formats like requirements.txt, environment.yml, install.R
- Mirror and archive all dependencies and sign the collection. Examples: {z3c.pypimirror, eggbasket, bandersnatch, devpi as a transparent proxy cache}, apt-cacher-ng, pulp, squid as a transparent proxy cache
- Produce a signed archive which includes all requisite software. (And host that download on a server such that data integrity can be verified with cryptographic checksums and/or signatures.) Examples: Docker image, statically-linked binaries, GPG-signed tarball of a virtualenv (which can be made into a proper package with e.g. fpm), ZIP + GPG signature of a directory which includes all dependencies
- Archive (1) the data, (2) the source code of all libraries, and (3) the compiled binary packages, and (4) the compiler and build userspace, and (5) the execution userspace, and (6) the kernel. Examples: Docker can solve for 1-5, but not 6. A VM (virtual machine) can solve for 1-5. OVF (Open Virtualization Format) is an open spec for virtual machine images, which can be built with a tool like Vagrant or Packer (optionally in conjunction with a configuration management tool like Puppet, Salt, Ansible).
When the application requires (7) a multi-node distributed system configuration, something like docker-compose/vagrant/terraform and/or a configuration management tool are pretty much necessary to ensure that it will be possible to reproducibly confirm the experiment output at a different point in spacetime.
A deep dive into the official Docker image for Python
One thing I've learned is that rather than use a docker-entrypoint.sh, most Linux software can be ran just using the `--user 1000:1000` or whatever UID/GID you want to use, as long as you map a volume that can use those permissions. It is a lot cleaner this way.
> Why Tini?
> Using Tini has several benefits:
> - It protects you from software that accidentally creates zombie processes, which can (over time!) starve your entire system for PIDs (and make it unusable).
> - It ensures that the default signal handlers work for the software you run in your Docker image. For example, with Tini, SIGTERM properly terminates your process even if you didn't explicitly install a signal handler for it.
> - It does so completely transparently! Docker images that work without Tini will work with Tini without any changes.
[...]
> NOTE: If you are using Docker 1.13 or greater, Tini is included in Docker itself. This includes all versions of Docker CE. To enable Tini, just pass the `--init` flag to docker run.
https://github.com/krallin/tini#why-tini
I didn't know it was included by default. I'll check it out, thanks!
There are Alpine [1] and Debian [2] miniconda images (within which you can `conda install python==3.8` and 2.7 and 3.4 in different conda envs)
[1] https://github.com/ContinuumIO/docker-images/blob/master/min...
[2] https://github.com/ContinuumIO/docker-images/blob/master/min...
If you build manylinux wheels with auditwheel [3], they should install without needing compilation for {CentOS, Debian, Ubuntu, and Alpine}; though standard Alpine images have MUSL instead of glibc by default, this [4] may work:
echo "manylinux1_compatible = True" > $PYTHON_PATH/_manylinux.py
[4] https://github.com/docker-library/docs/issues/904#issuecomme...
The miniforge docker images aren't yet [5][6] multi-arch, which means it's not as easy to take advantage of all of the ARM64 / aarch64 packages that conda-forge builds now.
[5] https://github.com/conda-forge/docker-images/issues/102#issu...
[6] https://github.com/conda-forge/miniforge/issues/20
There are i686 and x86-64 docker containers for building manylinux wheels that work with many distros: https://github.com/pypa/manylinux/tree/master/docker
A multi-stage Dockerfile build can produce a wheel in the first stage and install that wheel (with `COPY --from=0`) in a later stage; leaving build dependencies out of the production environment for security and performance: https://docs.docker.com/develop/develop-images/multistage-bu...
Interesting! I use miniconda extensively for local development to manage virtual environments for different python versions and love it. I hardly ever actually use the conda packages though.
I assume the main benefit of using these images would be if you are installing from conda repos instead of pip? Otherwise just using the official python images would be as good if not better
Edit: I guess if you needed multiple python versions in a single container this would be a good solution for that as well
Use cases for conda or conda+pip:
- Already-compiled packages (where there may not be binary wheels) instead of requiring reinstallation and subsequent removal of e.g. build-essentials for every install
- Support for R, Julia, NodeJS, Qt, ROS, CUDA, MKL, etc.
- Here's what the Kaggle docker-python Dockerfile installs with conda and with pip: https://github.com/Kaggle/docker-python/blob/master/Dockerfi...
- Build matrix in one container with conda envs
Disadvantages of the official python images as compared with conda+pip:
- Necessary to (re)install build dependencies and a compiler for every build (if there's not a bdist or a wheel for the given architecture) and then uninstall all unnecessary transitive dependencies. This is where a [multi-stage] build of a manylinux wheel may be the best approach.
- No LSM (AppArmor, SELinux, ) for one or more processes in the container (which may have read access to /etc or environment variables and/or --privileged)
- Necessary to build basically everything on non x86[-64] architectures for every container build
Disadvantages of conda / conda+pip:
- Different package repo infrastructure to mirror
- Users complaining that they don't need conda who then proceed to re-download and re-build wheels locally multiple times a day
Additional attributes for comparison:
- The new pip solver (which is slower than the traditional iterative non-solver), conda, and mamba
- repo2docker (and thus BinderHub) can build an up-to-date container from requirements.txt, environment.yml, install.R, postBuild and any of the other dependency specification formats supported by REES: Reproducible Environment Execution Standard; which may be helpful as Docker Hub images will soon be deleted if they're not retrieved at least once every 6 months (possibly with a GitHub Actions cron task)
Quite a few conda packages have patches added by the conda team to help fix problems in packages relying on native code or binaries. Particularly on Windows. If something is available on the primary conda repos it will almost assuredly work with few of any problems cross-platform, whereas pip is hit or miss.
If you’re always on Linux you may never appreciate it but some pip packages are a nightmare to get working properly on Windows.
If you look through the source of the conda repos, you’ll see all kinds of small patches to fix weird and breaking edge cases, particularly in libs with significant C back ends.
Here's the meta.yml for the conda-forge/python-feedstock: https://github.com/conda-forge/python-feedstock/blob/master/...
It includes patches just like distro packages often do.
The Consortium for Python Data API Standards
This actually strikes me as a huuuge waste of effort. I work with every one of the different technologies they mention every day.
The differences and idiosyncrasies are truly, truly not a big deal and really what you want is to allow different library maintainers to do it all differently and just build your own adapters over top of them.
This allows library developers to worry about narrow use cases and have their own separate processes to introduce breaking changes and features that deal with extreme specifics like GPU or TPU interop, heterogeneous distributed backed arrays, jagged arrays, etc. etc.
Let a thousand flowers blossom and a thousand different Python array and record APIs contend.
End users can write their own adapters to mollify irregularities between them, possibly writing different adapters on a case by case basis.
If any “standard” adapters gain popularity as open source projects, great - but don’t try to bake that in from an RFC point of view into the array & data structure libraries themselves. Let them be free / whatever they want to be. That diversity of API approach is super valuable and easily mollified by your own custom adapter logic.
Personally, I like how the Julia ecosystem coalesced around a Tables.jl package that gives some consistency to tabular data packages that choose to implement it. Row-oriented or column-oriented, doesn't matter! Using Tables.jl they can be interchangeable.
There’s a huge difference between “I personally like it when some libraries choose to adhere to a convention” vs “let’s bake this into every tool with a wide, bureaucratic shared RFC process.”
But I don't think people are forced into following right? There's still choice. But if I was developing something in Python, I would rather join than not join.
I’m saying as a consumer of these libraries, I don’t want joiners, I want lots of different APIs that make different trade-offs for each use case, and I will write (or use) adapters if I need to (that’s my responsibility as a consumer).
Just because I personally either do or do not get some value out of a shared API standard doesn’t mean you do or do not, so personal feelings of what one likes need to be set aside, so the service boundary is at the level of consumers choosing their own adapters and making their own trade-offs.
If lots of libraries sign up, we all lose. Much slower compliance-constrained development cycles, less freedom for libraries to break the shared API contract in ways many users would be super happy to abide for the sake of a faster / better feature. Instead of making these trade-offs library by library, case by case, it’s made for you with standardization compliance as the highest constraint, regardless of the user base for which that does / doesn’t work well.
This strikes me as a bit pessimistic. I think you are mostly against _bad_ standards than standards per se.
If the consortium does its job well it will produce a good standard that all the major libraries like: from a UX as well as a performance viewpoint.
If it produces a bad standard, all that will happen is that nobody will sign up. No libraries are going to be _forced_ to do anything.
That’s fair except I’d say even good standards can cause a big slowdown in feature releases, and not everyone values the adherence to the standard as much as the features. It still seems better to just spin out the standard into a set of optional adapters and let library maintainers and users pick their own trade-offs in terms of when to adopt breaking changes, how / whether to smooth over idiosyncrasies across multiple libraries.
Even a good standard has costs and this particular case does not seem like it has good arguments in favor of a wide standard, but many arguments against it.
No, it's easy for library maintainers to offer a compat API in addition to however else they feel they need to differentiate and optimize the interfaces for array operations. People can contribute such APIs directly to libraries once instead of creating many conditionals in every library-utilizing project or requiring yet another dependency on an adapter / facade package that's not kept in sync with the libraries it abstracts.
If a library chooses to implement a spec compatability API, they do that once (optimally, as compared with somebody's hackish adapter facade which has very little comprehension of each library's internals) and everyone else's code doesn't need to have conditionals.
Each of L libraries implements a compat API: O(L)
Each of U library utilizers implements conditionals for every N places arrays are utilized: O(U x N_)
Each of U library utilizers uses the common denominator compat API: O(U)
L < U < (L + U) < (U x N_)
Tech giants let the Web's metadata schemas and infrastructure languish
It's "langushing" and they should do it for us? It's flourishing and they're doing it for us and they have lots of open issues and I want more for free without any work.
Wow! Nobody else does anything to collaboratively, inclusively develop schema and the problem is that search engines aren't just doing it for us?
1) Search engines do not owe us anything. They are not obligated to dominate us or the schema that we may voluntarily decide to include on our pages.
We've paid them nothing. They have no contract for service or agreement with us which compels them to please us or contribute greater resources to an open standard that hundreds of people are contributing to.
2) You people don't know anything about linked data and structured data.
Here's a list of schema: https://lov.linkeddata.es/dataset/lov/ .
Here's the Linked Open Data Cloud: https://lod-cloud.net/
Does your or this publisher's domain include any linked data?
Does this article include any linked data?
Do data quality issues pervade promising, comparatively-expensive, redundant approaches to natural-language comprehension, reasoning, and summarization?
Here, in contributing this example PR adding RDFa to the codeforantarctica web page, I probably made a mistake. https://github.com/CodeForAntarctica/codeforantarctica.githu... . Can you spot the mistake?
There should have been review.
https://schema.org/ClaimReview, W3C Verifiable Claims / Credentials, ld-signatures, and lds-merkleproof2017.
Which brings us to reification, truth values, property graphs, and the new RDF* and SPARQL* and JSON-LD* (which don't yet have repos with ongoing issues to tend to).
3) Get to work. This article does nothing to teach people how to contribute to slow, collaborative schema standards work.
Here's the link to the GitHub Issues so that you can contribute to schema.org: https://github.com/schemaorg/schemaorg
...
"Standards should be better and they should pay for it"
Who are the major contributors to the (W3C) open standard in question?
Is telling them to put up more money or step down going to result in getting what we want? Why or why not?
Who would merge PRs and close issues?
Have you misunderstood the scope of the project? What do the editors of the schema feel in regards to more specific domain vocabularies? Is it feasible or even advisable to attempt to out-schema domain experts who know how to develop and revise an ontology or even just a vocabulary with Protegé?
To give you a sense of how much work goes into creating a few classes and properties defined with RDFS in RDFa in HTML: here's the https://schema.org/Course , https://schema.org/CourseInstance , and https://schema.org/EducationEvent issue: https://github.com/schemaorg/schemaorg/issues/195
Can you find the link to the Use Cases wiki (which was the real work)? What strategy did you use to find it?
...
"Well, Google just does what's good for Google."
Are you arguing that Google.org should make charitable contributions to this project? Is that an advisable or effective way to influence a W3C open standard (where conflicts of interest by people just donating time are disclosed)?
Anyone can use something like extruct or OSDS to extract RDFa, Microdata, and/or JSON-LD from a page.
Everyone can include structured data and linked data in their pages.
There are surveys quantifying how many people have included which types in their pages. Some of that data is included on schema.org types pages.
...
Some written interview questions:
> Which issues have you contributed to? Which issues have you seen all the way to closed? Have you contributed a pull request to the project? Have you published linked data? What is the URL to the docs which explain how to contribute resources? How would you improve them?
https://twitter.com/westurner/status/1291903926007209984
...
After all that's happened here, I think Dan (who built FOAF, which all profitable companies could use instead of https://schema.org/Person ) deserves a week off to add more linked data to the internet now please.
I think that might be fair, but when she makes org came out it pitched itself as trust us we will take care of things, so yeah they don’t owe us anything but track record matters for trust in future ventures by these search engine orgs
schemaorg/schemaorg/CONTRIBUTING.md https://github.com/schemaorg/schemaorg/blob/main/CONTRIBUTIN... explains how you and your organization can contribute resources to the Schema.org W3C project.
If you or your organization can justify contributing one or more people at full or part time due to ROI or goodwill, by all means start sending Pull Requests and/or commenting on Issues.
"Give us more for free or step down". Wow. What PRs have you contributed to justify such demands?
https://schema.org/docs/documents.html links to the releases.
Time-reversal of an unknown quantum state
T-symmetry https://en.wikipedia.org/wiki/T-symmetry > See also links to "reversible computing" but not the "time reversal" disambiguation page?
The "teeter-totter" thought experiment on that page is interesting to me in that it seems to illustrate how the future has more possibilities than the past. But it also occurs to me that the future possibilities are fractal with the past - ie. the toy could fall onto one of many other lower pedestals, teetering as it was before. In this way the "many arbitrary" possibilities could be perceived as anything but.
Electric cooker an easy, efficient way to sanitize N95 masks, study finds
An autoclave is what the electrical cooker is being used to emulate. A pressure cooker is a better option if you'll be on the road, camping, or otherwise without electricity. Researchers in Canada already proved several months ago that N95 masks could be sanitized this way:
Unfortunately the referenced NewsArticle does not link to the ScholarlyArticle https://schema.org/ScholarlyArticle :
"N95 Mask Decontamination using Standard Hospital Sterilization Technologies" (2020-04) https://www.medrxiv.org/content/10.1101/2020.04.05.20049346v... :
> We sought to test the ability of 4 different decontamination methods including autoclave treatment, ethylene oxide gassing, ionized hydrogen peroxide fogging and vaporized hydrogen peroxide exposure to decontaminate 4 different N95 masks of experimental contamination with SARS-CoV-2 or vesicular stomatitis virus as a surrogate. In addition, we sought to determine whether masks would tolerate repeated cycles of decontamination while maintaining structural and functional integrity. We found that one cycle of treatment with all modalities was effective in decontamination and was associated with no structural or functional deterioration. Vaporized hydrogen peroxide treatment was tolerated to at least 5 cycles by masks. Most notably, standard autoclave treatment was associated with no loss of structural or functional integrity to a minimum of 10 cycles for the 3 pleated mask models. The molded N95 mask however tolerated only 1 cycle. This last finding may be of particular use to institutions globally due to the virtually universal accessibility of autoclaves in health care settings.
The ScholarlyArticle referenced by and linked to by the OP NewsArticle is "Dry Heat as a Decontamination Method for N95 Respirator Reuse" (2020-07) https://pubs.acs.org/doi/full/10.1021/acs.estlett.0c00534 . Said article does not reference "N95 Mask Decontamination using Standard Hospital Sterilization Technologies" DOI: 10.1101/2020.04.05.20049346v2 . We would do well to record that (article A, seemsToConfirm, Article B) as third-party linked data (only if both articles do specifically test the efficacy of the given sterilization method with the COVID-19 coronavirus)
None of this is necessary for non-healthcare workers, since the virus will die off on a fabric mask in a few hours.
1) Not a single study has demonstrated that viable Sars-Cov-2 virus survives on porous materials in the real world
2) Even before Covid, it was known for decades (common medical knowledge) that human coronaviruses and flu viruses do not remain viable on porous materials for more than several hours.
That this common knowledge is not so common knowledge among the public is a failure of public health communication. The one or two alarmist studies showing that the virus "survives" X number of days don't reflect the real world because 1) the researchers literally directly douse or soak the surface with a huge viral load, and 2) the researchers usually only look for viral genetic material, not whether the virus can infect cells. Viability != detecting viral genetic material (same story for people - we shed viral genetic material long after we stop being contagious).
There is a reason the CDC has always said surface transmission is rare: https://www.nytimes.com/2020/05/22/health/cdc-coronavirus-to... there have been no documented cases of surface transmission: https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-si... Fortunately some in the media are catching on to the hygiene (security) theater: https://www.theatlantic.com/ideas/archive/2020/07/scourge-hy...
Citations for 1) and 2):
Most studies use unrealistic starting doses: https://www.thelancet.com/pdfs/journals/laninf/PIIS1473-3099...
Other human coronaviruses don't survive long on porous surfaces (gone by 6 hours): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7134510/. A hospital randomly collected patient and real surface swabs (including non-porous surfaces) for original SARS, a minority were PCR positive, none of the swabs were infectious (viral culture): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7134510/. Same story for flu (virus can be detected for days, but inactive and not viable after a few hours): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3222642/.
I could only find this study on Sars-Cov-2 that cultured the virus (still used a huge viral dose in a lab setting, not the real world). Even though they were able to culture the virus, only 1% of virus remains after 6 hours on a surgical mask, several orders of magnitude less for cotton clothing: https://www.medrxiv.org/content/10.1101/2020.05.07.20094805v...
This long post was originally meant to be a reply to someone asking for a citation; it's yet another example of how much effort is required to combat misinformation.
"Interim Recommendations for U.S. Households with Suspected or Confirmed Coronavirus Disease 2019 (COVID-19)" https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-si... :
> On the other hand, transmission of novel coronavirus to persons from surfaces contaminated with the virus has not been documented. Recent studies indicate that people who are infected but do not have symptoms likely also play a role in the spread of COVID-19. Transmission of coronavirus occurs much more commonly through respiratory droplets than through objects and surfaces, like doorknobs, countertops, keyboards, toys, etc. Current evidence suggests that SARS-CoV-2 may remain viable for hours to days on surfaces made from a variety of materials. Cleaning of visibly dirty surfaces followed by disinfection is a best practice measure for prevention of COVID-19 and other viral respiratory illnesses in households and community settings
Fed announces details of new interbank service to support instant payments
If anyone from the FED is reading this I have this advice: don't use ISO20022! (I've been working on the brazilian instant payments project, known as PIX). It's an overly-complicated-try-to-solve-everything standard that promises interoperability but only delivers pain and misery. Since each country has its own standards for person and account identification, in dealing with these local realities, the ISO20022 adds quite a lot of complexity. But all this cost does not result in interoperability. In fact, each payment scheme end up with a localized version of the standard, incompatible with others. What could have been a clean API, ends up a mess.
What alternative do you suggest? Keeping in mind all the parties that will be involved, besides your project.
Create a fit to purpose, simpler, protocol using a good interface specification language (OpenAPI or protocol buffers). Maybe borrow concepts and patterns from the ISO standard, but not adopt it. It will not deliver its promise of interoperability. It might actually make it harder to interoperate with other payment schemes.
Interledger Protocol (ILP, ILPv4).
Interledger Architecture:
https://interledger.org/rfcs/0001-interledger-architecture/#... :
> For purposes of Interledger, we call all settlement systems ledgers. These can include banks, blockchains, peer-to-peer payment schemes, automated clearing house (ACH), mobile money institutions, central-bank operated real-time gross settlement (RTGS) systems, and even more.
[...]
> Interledger provides for secure payments across multiple assets on different ledgers. The architecture consists of a conceptual model for interledger payments, a mechanism for securing payments, and a suite of protocols that implement this design.
> The Interledger Protocol (ILP) is the core of the Interledger protocol suite. Colloquially, the whole Interledger stack is sometimes referred to as "ILP". Technically, however, the Interledger Protocol is only one layer in the stack.
> Interledger is not a blockchain, a token, nor a central service. Interledger is a standard way of bridging financial systems. The Interledger architecture is heavily inspired by the Internet architecture described in RFC 1122, RFC 1123 and RFC 1009.
[...]
> You can envision the Interledger as a graph where the points are individual nodes and the edges are accounts between two parties. Parties with only one account can send or receive through the party on the other side of that account. Parties with two or more accounts are connectors, who can facilitate payments to or from anyone they're connected to.
> Connectors [AKA routers] provide a service of forwarding packets and relaying money, and they take on some risk when they do so. In exchange, connectors can charge fees and derive a profit from these services. In the open network of the Interledger, connectors are expected to compete among one another to offer the best balance of speed, reliability, coverage, and cost.
ILP > Peering, Clearing and Settling: https://interledger.org/rfcs/0032-peering-clearing-settlemen...
ILP > Simple Payment Setup Protocol (SPSP): https://interledger.org/rfcs/0009-simple-payment-setup-proto...
> This document describes the Simple Payment Setup Protocol (SPSP), a basic protocol for exchanging payment information between payee and payer to facilitate payment over Interledger. SPSP uses the STREAM transport protocol for condition generation and data encoding.
> (Introduction > Motivation) STREAM does not specify how payment details, such as the ILP address or shared secret, should be exchanged between the counterparties. SPSP is a minimal protocol that uses HTTPS for communicating these details.
[...]
GET /.well-known/pay HTTP/1.1
Host: example.com
Accept: application/spsp4+json, application/spsp+jsonShrinking deep learning’s carbon footprint
"Unlearning" is one algorithmic approach that may yield substantial energy consumption gains.
With many deep learning models, it's not possible to determine when or from what source something was learned: it's not possible to "back out" a change to the network and so the whole model has to be re-trained from scratch; which is O(n) instead of O(1.x).
Despite Pyolite has a miserable performance (20MB of downloads), the overall project direction is correct.
I said this already 10 years ago: We don't need more cloud computing but need to empower users end devices again. Jupyter is typically operated on powerful notebooks and not on mobile devices.
Isn't that not just Java all over again, but this time with JavaScript?
There are crucial differences between Java applets and JS.
- Applets tried to render their own GUI, Wasm doesn't and defers to the browser.
- applets needed a big, slow to start and resource hungry VM. Wasm is running in the same thread your JS is also running in, it's light, and loads faster than JS
- Java and flash were plugins, which needed to be installed and kept up to date separately. Wasm is baked into your browser's JS engine
- Wasm code is very fast and can achieve near native execution speeds. It can make use of advanced optimisations. SIMD has shipped in Chrome, and will soon in Firefox
- The wasm spec is very, very good, and really quite small. This means that implementing it is comparatively cheap, and this should make it easy to see it implemented by different vendors.
- Java was just Java. Wasm can serve as a platform for any language. See my earlier point about the spec
So it's apples and oranges. The need to have something besides JS hasn't gone away, so their use cases might be similar. The two technologies couldn't be more distinct, though.
You must view the browser with JS and WASM as a unit.
The browser renders it's own GUI too, it's not OS native
The browser uses lots of resources too.
The browser is kind of a plugin to the OS and must be updated separately.
Java nowadays is pretty fast too.
Java VM serves a platform for multiple languages like Scala, Kotlin, Clojure.
Let's face it, the browser is the new JVM and a soon it gets the same permissions like the JVM to access the file system and such, we get the same problems.
From https://news.ycombinator.com/item?id=24052393 re: Starboard:
> https://developer.mozilla.org/en-US/docs/Web/Security/Subres... : "Subresource Integrity (SRI) is a security feature that enables browsers to verify that resources they fetch (for example, from a CDN) are delivered without unexpected manipulation. It works by allowing you to provide a cryptographic hash that a fetched resource must match."
> There's a new Native Filesystem API: "The new Native File System API allows web apps to read or save changes directly to files and folders on the user's device." https://web.dev/native-file-system/
> We'll need a way to grant specific URLs specific, limited amounts of storage.
[...]
> https://github.com/deathbeds/jyve/issues/46 :
> Would [Micromamba] and conda-forge build a WASM architecture target?