Hello Printavo Users,
Earlier today the Printavo platform experienced a significant degradation of service which resulted in the inability of many of our customers to access the platform in a timely manner. The incident occurred from 10:51am to 12:15pm Central, followed by a much smaller but still significant increase in request queue wait time from 1:07pm to 1:37pm central time. The root cause of these incidents stemmed from an issue with our firewall which incorrectly identified our marketing page as down rerouting marketing page traffic (malicious and otherwise) through the main platform's front door. The end result was that this deluge of requests ended up taking the slots of legitimate requests to the Printavo platform. To make matters worse the marking site experienced slowness of its own which exacerbated the issue.
As a result of the above, we have opened a ticket with our firewall provider and have set a more aggressive timeout for marketing content for the time being. This means that in the future should something like this occur again we will favor serving requests to the platform over those of the marketing pages. This strategy was the reason why the second degradation was both shorter and less intense than the first. That said as a platform which helps our customers run their business neither is ideal nor acceptable. We have historically taken great pride in the performance of Printavo as well as its uptime. Starting with the last week of April through today we have experienced numerous attacks as well as scaling challenges as the number of customers has grown. We apologize for the frustration these outages have caused and are committed to keeping extra resources on the situation until it fully stabilizes.
Our engineering team is continuing to monitor the situation to ensure that should anything new come up we can promptly respond to keep our customers up and running. Again, we apologize for the disruption to your business. When your business is disrupted so is ours and as we stated above we don't take either lightly. Thank you for your continued support.
Sincerely,
Piotr Biegaj
Inktavo CTO