Wednesday, February 6, 2013

Large Reporting Jobs with Amazon RDS Read Replicas (MySQL)

  I recently came across a project where a monthly report was overwhelming a small multi-AZ MySQL RDS instance. For 29 days of the month the small instance was sufficient, but running the monthly report would not only take days to complete, but it would bog down the entire website while it was running.

  First, a little background. The website in question is a public internet facing website. It consists of two small web server instances behind an Elastic Load Balancer, a small multi-AZ RDS instance, and a small instance used for running administrative processes and reports.

   In a non-cloud environment the only answer would be to either upgrade the database server, or add a second database server dedicated for reporting jobs. Luckily this website was running in AWS.

  With AWS we could easily upgrade the database to a larger instance to support the production load and the monthly report, but the extra capacity provided by the larger instance would be expensive and go unused for all but one day a month. We could manually create a read replica, allowing the reporting job to run against another small instance, but that the job would still take days to complete, and again we'd be paying for unused resources for most of the month. 

  The answer we came up with was simple - we modified the reporting job to take advantage of the AWS API and launch a huge read replica of the database. The job flow went like this:
  1. Launch High Memory Double Extra Large read replica of the RDS instance*
  2. Once the read replica was online execute our monthly report
  3. Destroy read replica once the job completes**
  4. Email report results to interested parties
  This solution is the best of both worlds - The monthly report gets to take advantage of the resources available in the larger database instance to complete in hours instead of days, and the customer only pays for the resources they actually need. Instead of paying several hundred dollars per month to upgrade the multi-AZ instance to support both workloads, the customer ended up paying about $10 a month for better results.

  The moral of the story is that while it's totally possible to treat cloud resources just like traditional servers in a data center, you're usually doing yourself a disservice by doing so. By taking advantage of the extra flexibility and features built into most cloud platforms you can not only save money, but also provide an all around better experience for yourself and your end users.

18 comments:

  1. I admire the valuable information you offer in your articles. I will bookmark your blog and have my friends check up here often. I am quite sure they will learn lots of new stuff here than anybody else! Regards aws jobs in hyderabad.

    ReplyDelete
  2. It's very useful blog post with inforamtive and insightful content and i had good experience with this information. We, at the CRS info solutions ,help candidates in acquiring certificates, master interview questions, and prepare brilliant resumes.Find top Salesforce admin interview questions in 2020.
    These Salesforce developer interview questions are highly helpful in 2020. You can read these Salesforce lightning interview questions and Salesforce integration interview questions which are prepared by industry experts.

    ReplyDelete
  3. Gone through this wonderful coures called Salesforce Certification Training in Dallas who are offering fully practical course, who parent is Salesforce Training in USA and they have students at Salesforce Training classes in Canada institutes.

    ReplyDelete
  4. You can join the free orientation at Workday training institutes in Bangalore and they located at Workday training institutes in delhi. It is highly recommendable for you to enroll at Workday training institutes in UK and you can have a great market for jobs in USA so join this wonderful Workday training institutes in USA and in India this Workday training institutes in Noida is really best institute.

    ReplyDelete
  5. I've also experienced it twice in the software industry when I wasn't enjoying my work, wasn't passionate about it, and didn't believe in the company. In these instances, somebody else liberated me! Salesforce training in Hyderabad

    ReplyDelete
  6. Email Us:- inkdrop121@gmail.com

    We're the producers of legal documents and we provide documents like license, passport, visa, ssn, citizenship certificate and many more. If you are interested or for more information please do not hesitate to get in touch with us via Email Us:- inkdrop121@gmail.com

    Website Link: linkedin.com/in/ink-drop-2a3a39209

    ReplyDelete
  7. You need personal informations from companies,family and friends that will better your life and you need easy access without them noticing just contact james or you’re financially unstable or you have a bad credit score, he will solve that without stress,he and his team can clear criminal records without leaving a trace and can also anonymously credit your empty credit cards with funds you need,all these are not done free obviously but I like working with James and his team cause they keep you updated on every step taken in order to achieve the goal and they also deliver on time,I tested and confirmed this I’m still happy on how my life is improving after my encounter with him ,you can send a mail to premiumhackservices@gmail.com

    ReplyDelete
  8. Simply appreciating your work and thinking about how you dealt with this blog so well. It's striking to such an extent that I can't stand to not experience this important data at whatever point I surf the web! SEO expert Singapore

    ReplyDelete
  9. We are most reliable and efficient supplier/distributor of all kinds of commonly used pain killers, anxiety meds, depression meds, sleeping/sex aid pills with safe and quick delivery to any part of the world without custom issues.. We do worldwide shipping to any clear address.
    Customer satisfaction is our greatest concern. Shipping is 1-3 working days depending on your country.
    For more inquiries on price, purchase and shipping, get back to us at; reimannbrigit.9@gmail.com or whatsApp:+1(281) 393-7628.

    Get all illicit and illegal drugs delivered to you all over the world and anywhere you want; We have the below drugs and more in stock;
    Heroin, Adderall,
    Cocaine,
    Marijuana/Cannabis,
    MDMA,
    PercocetNorco 325 mg / 5 mg,
    Vicodin 500 mg / 5 mg,
    Viagra 100 mg,
    Xanax 2 mg longbars,
    Seconal (Secobarbital Sodium Capsules) 100 mg,
    Nembutal (In pill, liquid and powder form),
    Quaalude (lemmon 714 300mg),
    Ketamine Crystal,
    Dilaudid 8 mg,
    Adderall 30 mg,
    Oxycontin 20 mg / 80 mg,
    Klonopin 0.5 mg,
    Ritalin 10 mg,
    Actavis: promethazine/codeine syrup,
    Ambien ( Zolpidem, Stilnox) 10 mg,
    Clonazepam 2 mg ( Rivotril),
    Lorazepam 2.5 mg ( Ativan),
    Roxicodone 15 mg, 30 mg,
    Hydro 10 mg / 500 mg,
    Percocet 10/325mg,
    Opana 10 mg / 40 mg,
    Subutex - Suboxone,
    Fentanyl patches,
    Diazepam Roche 10mg,
    Alprazolam/Xanax 0.5mg,
    Zopiclone 7.5mg,
    Diazepam Shalina 10mg,
    Diazepam Teva 5mg,
    Zolpidem 10mg,
    Tramadol 200mg,
    Ketamine liquid 50mg/10ml,

    Our services are as seen below;
    - Stealth Packages
    - Vacuum sealed thick plastic
    - No prescription required
    - Express prior overnight or next day delivery
    - Fast and Reliable delivery #Tracking Available!
    - Packing is highly discreetly sealed, wrapped with an aluminum foil, safe and secure
    - No signature required upon arrival of parcel
    - 100% customer care and support.
    - Money back guarantee
    - Offers Reshipment if package do not gets to location
    - We offer door to door delivery services
    * We are reliable, and we offer the best prices and services for our products.
    * We offer safe and discreet overnight shipping (24/h) to clients within the
    USA, and 3-4 business days shipping to clients anywhere in the world.
    * We also offer the best discount for bulk purchase on any of our products.
    * We offer the best of services to all our clients, and make sure they are treated like family.
    * We offer a FULL REFUND on any package that doesn't make it to its destination.
    Contact us:
    WhatsApp: +1(281) 393-7628
    Email: reimannbrigit.9@gmail.com

    ReplyDelete
  10. I am so delighted I found your weblog, I really found you by accident,
    while I was researching on Bing for something else, Regardless I am here now and would just like to say thank you for a fantastic post and a all round entertaining blog (I also love the theme/design), HOW TO GET A FAKE DRIVERS LICENSE THAT WORKS FROM THE DMV I don't have time to go through it all at the minute but I have bookmarked it and also added your RSS feeds, so when I have time I will be back to read a lot more, Please do keep up the excellent job.

    ReplyDelete