Apologies for the ongoing nature of these issues. The problems are down to load/memory and the transparent proxy platform is being upgraded on Sunday. Updata tell me they are confident from that moment forward, everything will be okay. Like you, I really hope that is indeed the case and I’ll be glad to put this to bed. We do however have a day (Friday) to get through first and I have to be honest in that we could once again experience performance issues. With this in mind, Updata have put a limit on the amount of sessions the platform can deal with simultaneously. So for tomorrow, the transparent proxy platform can only deal with 200,000 concurrent sessions, any more and the proxy will not accept new connections. This should stop the server ‘failing over the edge’ but will no doubt bring problems of its own.
I don’t really know what else to say, you’ve probably had enough of words and posts and just want action… which I completely understand. Senior management at HfL, HCC and Updata are aware of the problems. In the next contract (with Updata or someone else) there’s going to be greater focus on resiliency testing to try and pick up on these things before they cause performance related issues.
Further to my earlier post, maybe it is a good idea to post an update. When the transparent proxy fails, it happens so quickly that the network monitoring tool does not have chance to warn Updata. The only way to get the service back online is to despatch a field engineer to the data centre (in Slough) to reboot the hardware.
With the pending upgrade to fix this properly scheduled for this weekend, it leaves us in a vulnerable position between now and then. With this in mind, Updata are going free up internal resource and make sure a field engineer is close to the data centre at all times. I completely agree this is a very reactive action but as this weekend’s work is too large a job to do in the evenings, we are on the verge of running out of options.
Any questions, thoughts, please contact me in the usual way. Thanks for your patience – it is not being taken for granted.
Apologies but as you may know the transparent proxy is down again today. There is an upgrade on the platform scheduled for this coming weekend but this should have been carried out over the summer. Updata are confident that this will fix the problem (it will bring added functionality too but more about that in due course). Very frustratingly Updata couldn’t manage this platform upgrade over the summer after previously assuring us they would get this done. Please note that the downtime is impacting schools and HCC Corporate sites, so it isn’t just me complaining to Updata – it is HCC Corporate too. This is particularly bad timing for Updata as the procurement process is under way to see who HfL’s broadband provider is 2017 onwards!!
The obvious question you may have is whether the transparent proxy will keep dropping out between now and the weekend. At present, I can’t provide any reassuring answers. Lots of questions and escalations are occurring behind the scenes and I will update you all shortly. Please do not think that neither myself, HfL or HCC are anything other extremely unsatisfied with the situation.
Any queries – please get in touch,
Some schools may be experiencing issues with the transparent proxy network. Updata have logged a Priority 1 incident and are urgently dealing with. Just to remind you all the platform is being upgraded in 8 days times. Thanks for your understanding.
There are problems at present and Updata are dealing with. Apologies for the inconvenience. FYI – an upgrade to the platform has been scheduled for October 1st.