PlantX: Ready for the future! 2020 was the year interrupted, and so we used the time to get PlantX ready for the future! As we enter our third decade of operations (yes, it's been 20 years), we made some underlying technical changes necessary to pave the way for the next decade.
In short, with isolation time on our hands, we spent nearly 2,000 hours upgrading our services and servers.
- We upgraded the database technology used by the nearly 600 operations that make up PlantX
- We moved our cloud services to Amazon's 1.6 trillion dollar cloud hosting services.
New Database Tech
Every time someone adds, changes or deletes information in PlantX it requires a database operation. By upgrading the underlying database, the service can now process 10 times as many operations every second! Furthermore, it allows us to process many of those operations simultaneously! This reduces, or eliminates any congestion during daily operations.
In order to support the new database features, it required that we rewrite nearly 600 server operations. To ensure a seamless transition, we also wrote a test for each one to check the integrity of the upgrade.
For 20 years we've maintained our own server equipment, Internet connections and the firewall and security between the two. We budget for regular server upgrades every 3 to 4 years and replace storage devices every 18 months. This time however, rather than upgrade our server equipment, we have moved our services to Amazon's trillion dollar network.
Four years ago we reviewed Amazon's offering but the costs were prohibitive. Last fall we checked again and the costs had fallen significantly, nearly 80%. Although moving our cloud to Amazon's services will triple our typical server budget, we feel the added benefits from Amazon's infrastructure and services will net an equal return.
To name just a few of theses benefits:
- Faster! Our fiber network is 150 mbps (megabits per second), while Amazon's network is 10,000 mbps. That's 10 gigabits! Amazon servers can “talk” to one another across the country 10 times faster than the computers can talk within your office.
- Quicker! Not only can Amazon's network move more information, it can respond more quickly to small requests. Every add, change or delete you perform in PlantX is only limited by the speed of your Internet connection.
- Everywhere! Amazon's network is a global network and designed to serve data all over the globe. Known as an “edge network”, your data is served from several locations around the world so your website always appears nearby. For example, the websites are hosted in Amazon's Oregon facility, but east coast customers will be served simultaneously by Amazon's east coast servers.
- Availability! Never another power outage to interrupt service. We had a couple of outages last year that exceeded our battery backup's one-hour run time, and the last one in November actually destroyed the battery backup. Amazon's facilities have redundant power sources and backup generators to avoid any such interruptions.
- Reliability! We use multiple redundant disks working together (RAID) to ensure the safety of your data. But Amazon operates in a whole new stratosphere when it comes to data storage and databases. Their services are broken into regions and availability zones to provide even greater reliability.
- Backups vs. Snapshots! Our local servers performed daily backups, where the database and other critical files are copied to an off-site location every night. Amazon, however, performs complete snapshots of the entire storage drive every day. While we had backups of every critical file, Amazon makes a daily snapshot of everything. And you can never have enough copies, so we will continue to do our off-site backups.
- Watching it all! We now get real-time graphs for every single operating device. We can see how busy each server is and where there might be a bottleneck. We can see how much data is moving over the network and to each storage device. We have one dashboard where we can keep our eye on the PlantX pulse.
- Ready to grow! As PlantX grows and we need more resources, we simply execute a few commands. More disk space, done. More processing power, done. More servers, done. To put this into perspective, just one of Amazon's computer processors costs more than our entire rack of servers. Amazon is more than happy to sell us more of anything we need ($$$). At the same time, we don't pay for what we don't use. On balance, Amazon will accommodate our growth requirements with predictable, and often declining costs.
Minor Upgrades/Bug Fixes
- Minor web cart fix
- Minor window title fix
- Order totals fix
Pallet % to Per Pallet The way pallet % was calculated, using a percentage, was too easily skewed (either too many or too few depending on rounding). This update changes the current pallet % to the total number of the catalog items that will fit on the pallet. For example 50% changes to 2, 25% changes to 4, etc. When adding new Per Pallet data, you now choose the quantity of a catalog item that fits on one pallet.
This is one of the largest technical upgrades in the past 10 years. It has taken over a year of programming to bring it altogether. It centers on making PlantX faster and more reliable in daily use.
There are an average of 200 user connections to PlantX daily. Added to that the hundreds of web users that come and go. Included are upwards of 5,000 reports run each week. Demand for PlantX is growing and these changes were required to ensure PlantX can meet the continued increase well into the next decade.
- Durable Connections
- Improved Database
- Data Integrity
- No Hesitations
Durable Connections As the Internet
has gotten faster, it has come at the cost of reliability. When PlantX was started in 2001,
the best connections you could get for your office were modems, DSL and occasionally cable.
But once your computer was connected to the Internet, you could connect to PlantX and that
connection was good all day. The connections were slower, but also steady and consistent.
Entering the modern era, connections are MUCH faster but they’ve become unreliable. Over the past few years we noticed some customers getting disconnected randomly and more often. This was unacceptable and we started looking deeper into the cause. It turns out modern network routers are sub-standard.
We solved the first problem by helping some members install professional routers. These are much better than the ones you get at Best Buy, but not as plug-n-play friendly. They made significant improvements, but we can’t afford to install equipment for everyone.
The second problem was much more subtle and nefarious; connections were deliberately dropped or blocked by their Internet Service Provider (ISP). The connections were dropped to thwart file sharing. So if you weren’t browsing the Internet or streaming a movie, your connection was vulnerable. Since PlantX didn’t look like someone browsing the Internet, the connection was expendable.
This required months of work to re-engineer the communications to use the same protocol used by web browsers. In late May, we released those updates and PlantX is now indistinguishable from any other web traffic. To the Internet providers, PlantX just looks like a busy website.
Improved Database In early June, we
upgraded the database software that underpins the PlantX server. The database upgrade included
some long awaited features that could improve PlantX performance. We applied the features in a
set of trial transactions. The trials were successful, showing a marked improvement in
At that time we decided to upgrade all 500+ transactions (e.g. add order, change order item, add product, etc.) We then spent the next 90 days straight modifying each operation to comply with the new database standards. And to ensure each transaction performed to specification, we wrote a test program for every operation!
As for performance...before we made the changes, our complete test took 70 seconds to run. Now we have nearly doubled the number of tests and it runs in under 15 seconds!
Multiprocessing That’s tech-speak
for doing two things at once. Modern computers have the ability to do many things at once
because they have multiple “cores” and we try to keep all the cores busy. But sometimes you
can’t do two things at once. For example, a truck can’t deliver to Chicago and New York at the
very same time. It can however deliver to Chicago and then move on to New York.
The same limitation applied to PlantX transactions. To guarantee integrity we could only process one at a time because of database limitations. But with the upgraded database, and some added engineering on our part, PlantX can now handle most transactions simultaneously!
As for performance...the numbers speak for themselves. We run all 166 test units, each with an average of 4 tests, that trigger over 3,500 transactions that all complete in under 9 seconds!
Data Integrity One of the marked improvements of the database is the data integrity. Integrity means that the database won’t let you delete products if they’re used by order items, or you can’t add a supply without a valid crop. Previously, we had to manually program each step to preserve integrity. It meant for every change request, we had to read copious amounts data to make sure the change was permitted and accurate. Now, the upgraded database enforces the integrity, and because it holds all the data, the check is immediate. This makes all operations much faster and the data is kept in check.
No Hesitations When the Internet was
slower, your computer couldn’t afford to make extra trips back and forth to the server. So
each call to the server was optimized by hesitating slightly to see if any more data was ready
for pickup. This is the city bus analogy; the bus doesn’t take off to the next stop just
because one passenger boards. It hesitates to allow more passengers, or in this case more data
to get on board.
But today’s Internet is much quicker and the server is much faster at gathering data for the bus. So we took out the hesitation! The hesitation was only 1/20th of a second, but eliminating the delay had a significant impact on other processes. For example, when we run 500 tests back to back, 1/20th of a second adds up to 25 seconds of wasted time.
As for performance...if you do something simple, for example clicking to toggle a supply listed, it will be immediate. Now of course the responsiveness still depends on the speed of your Internet, but the transaction itself no longer hesitates.
Can you hear me now? This PlantX upgrade includes a new feature that attempts to reconnect to the service if there is an interruption between you and the service. PlantX will effectively redial and take up where it left off. No more PlantX closures when your Internet connection hiccups!
PlantX started back in 2001, the days when we accessed the Internet with modems and DSL (a faster modem). Back then, a connection meant you were connected as long as the modem was running. Today we have cable, fiber and wireless connections which are magnitudes faster but they've become unreliable, and in some cases hostile. Some Internet providers deliberately kill connections! They expect their customers to only browse the web and stream videos. Any uses outside of that for business, disconnects are commonplace.
This upgrade adapts to these new conditions by temporarily suspending operation while it reconnects to the server. Previously, if the connection was interrupted, PlantX closed and you had to restart. Now PlantX will detect interruptions and wait to recover the connection. It waits up to two minutes before closing. While interrupted, or even completely disconnected, you will see a new connection message shown here.
While the connection message displays, PlantX runs some simple diagnostics to help give you (and us) clues to how your computer's connection is performing. It tests the local computer and then well known sites on the Internet, then our network, and finally the PlantX service. If any tests return with a red X, it can reveal why the connection is down.
Give it a try! If you dare...
If you want to see this new feature in action, start PlantX and then unplug your computer from the Internet! I don't really recommend that you do it, but it does work. This is exactly how we test this feature! Just remember to plug back in before two minutes.
A note for those that use Broker Servers.
If your office has a Broker Server installed, and the Broker's Internet connection is interrupted, you will get a similar window showing the broker is disconnected. It doesn't show detailed diagnostics, but it does let you know that service to the Broker is interrupted. You may want to check your Internet connection, and if you see these often, let us know.