What is Cloudlytics ?

Cloudlytics is a Software as a Service (SaaS) Big Data platform that helps customers to analyze and process your Amazon Web Services (AWS) Simple Storage Service (S3) and CloudFront log files.

  • Cloudlytics generates detailed and insightful reports of access patterns of your online assets rendered using AWS S3 & CloudFront.
  • Cloudlytics is simple to configure service and is a pay-as-you-go service with no upfront costs.


 What can I do with Cloudlytics ?

Cloudlytics processes AWS S3 and CloudFront log files using AWS EMR and RedShift.

Cloudlytics provides:

  • Detailed analysis of your CloudFront and S3 access patterns
  • Dynamic graphs to get a 360 degree perspective
  • Cost saving tips and tricks
  • Spam Reports

 How to get started with Cloudlytics ?

Getting started with Cloudlytics is straightforward.

You can get started in 3 simple steps:

  • Register on Cloudlytics, which is free
  • Configure a service (AWS CloudFront distribution / S3 bucket)
  • Analyze & Generate Reports.

You can find detailed help documents once you register with Cloudlytics.


 What do I need to get started with Cloudlytics ?

Here are some pre-requisites for using Cloudlytics:

  • You should be using AWS CloudFront or S3 to store and distribute content.
  • Logging should be enabled for your CloudFront distribution or S3 bucket
  • To start using Cloudlytics immediately, you should have at least one day of logs generated in your S3 buckets.

 How does Cloudlytics work ?

Once you register with Cloudlytics and you meet the pre-requisites mentioned earlier, you need to create a user "Cloudlytics" with the right IAM policy.You can find this policy in the help section of Cloudlytics.

Once a service is configured, Cloudlytics will fetch the log files generate yesterday. If there are files present for yesterday, then Cloudlytics will process these files and generate reports. If there are no files present for yesterday, then you will see a message "No Logs to Process" in front of your distribution.

In either case, Cloudlytics will process your log files daily from next day.

Why previous day logs ?

AWS policy states that CloudFront logs files may take up to 24 hours from the time of access to generate and store log files in your defined S3 buckets.

Similarly for S3, the AWS policy states that S3 logs files may take up to few hours from the time of access to generate and store log files in your defined S3 buckets.

Cloudlytics processes previous day log files so that we have maximum log files for a particular day for sampling when you start using the service.


 Does Cloudlytics process log files in real time ?

No, Cloudlytics does not process log files in "real time". The logs are processed at an interval of 24 hours.


 Do I need to enter my credit card details to use Cloudlytics ?

You can sign-up for the Cloudlytics free tier that does not require you to enter your credit card details. In case you want to process logs beyond the free tier limit, you will have to enter your credit card details.


 How many CloudFront distributions or S3 buckets can I configure with Cloudlytics ?

The number of CloudFront distributions or S3 buckets you can configure depends on your subscription plan. For more details on the subscription plans, please visit the pricing section.


 I no longer want to process logs for a particular CloudFront distribution or S3 bucket, how do I stop the process for a particular end-point ?

You can deactivate a particular end-point to stop processing logs from your account dashboard. For more information, please visit the help section in your account.


 What should I do I want to discontinue using Cloudlytics ?

Cloudlytics is a pay-as-you-go service with no upfront commitments. If you wish to discontinue using Cloudlytics, you can raise a request to deactivate your account.

You will not receive any refunds if you have already paid for a monthly subscription.


 How will I deactivate/delete my account ?

To deactivate/delete account you need to contact us offline using the support form on the website or write us at support@cloudlytics.com.



 How to configure Cloudlytics for my AWS CloudFront distribution or S3 bucket ?

You need to register with Cloudlytics and meet the following pre-requisites before you can configure your services.

  • You should be using AWS CloudFront or S3 to store and distribute content.
  • Logging should be enabled for your CloudFront distribution or S3 bucket.
  • To start using Cloudlytics immediately, you should have at least one day of logs generated in your S3 buckets.

To configure a new service,

  • Click on "Add Service" on the left sidebar.
  • You will find details about IAM policy and the steps to create an IAM user in your AWS management console.
  • Create a user "Cloudlytics" using your AWS management console, steps to create IAM user in AWS management console following the steps mentioned on the screen.
  • Once you have created a user, you have to select the type of service like "CloudFront Download", "CloudFront Streaming" or "S3".
  • You have to enter ACCESS KEY and SECRET ACCESS KEY of the user you created for Cloudlytics. You can find the ACCESS KEY and SECRET ACCESS KEY for your AWS account in your AWS management console.
  • If the ACCESS KEY and SECRET ACCESS KEY you have entered are correct, Cloudlytics will fetch the list of buckets and distributions associated with your AWS account.
  • Select the bucket or distribution you want to configure.
  • You will see an Action button with one of these 3 messages, "Start Analyzing", "Can't Analyze" or "Configured with Cloudlytics".
  • Click on "Start Analyzing", Cloudlytics will process log files for the selected bucket or distribution.
  • The message "Can't analyze" means that logging is not enabled for that bucket or distribution. You need to enable logging for the bucket or distribution and try again.
  • The message "Configured with Cloudlytics" means that particular bucket or distribution is already configured with Cloudlytics. Check with your admin or team members to verify if anyone else from your organization has signed up for the service.

 How to create IAM User and set IAM Policy for Cloudlytics in your AWS management console ?

You can create an IAM user and set the IAM policy for Cloudlytics using the following steps,

  • "Login" to your AWS Console.
  • Choose "Create New Users" in the IAM Console.
  • Make sure the checkbox "Generate an access key" for each User is selected.
  • After creating the user, select "Show User Security Credentials".
  • Select the newly created user in the list and choose the "Permissions" tab.
  • Select "Attach User Policy".
  • Choose "Custom Policy" option.
  • Copy "Custom Policy" from Cloudlytics and paste it.
  • After Pasting, replace "log-bucket-name" in "Custom Policy" with the bucket name that contains logs.
  • Select "Apply Policy".

 How Cloudlytics works after configuring AWS CloudFront/S3 ?

Once a service is configured, Cloudlytics will fetch the log files generate yesterday. If there are files present for yesterday, then Cloudlytics will process these files and generate reports. If there are no files present for yesterday, then you will see a message "No Logs to Process" in front of your distribution.

In either case, Cloudlytics will process your log files daily from next day.

Why previous day logs ?

AWS policy states that CloudFront logs files may take up to 24 hours from the time of access to generate and store log files in your defined S3 buckets.

Similarly for S3, the AWS policy states that S3 logs files may take up to few hours from the time of access to generate and store log files in your defined S3 buckets.

Cloudlytics processes previous day log files so that we have maximum log files for a particular day for sampling when you start using the service.


 What is the frequency of log processing in Cloudlytics ?

Cloudlytics processes logs once in every 24 hours.


 How to get your historical logs processed by Cloudlytics ?

To process your historical log files, you can send a not to support@cloudlytics.com.

Any historical processing will be charged an extra amount depending on the cumulative size of the historical log files. Support team will confirm the amount and raise an invoice before processing the historical log files.


 How to activate or deactivate CloudFront or S3 endpoints configured with Cloudlytics ?

To activate/deactive an endpoint, there is an option on Dashboard to do it. If an endpoint is in "active" state, there will be option to "deactivate" it and vice versa.


 How to change the AWS credentials with the expired one ?

If you have changed AWS credentials for Cloudlytics user in your AWS account then you can update those credentials in Cloudlytics by clicking on "Settings"->"Update Credentials" on the left sidebar.

Cloudlytics will use the new credentials to access log files the next time.


 How will I delete an endpoint ?

You cannot delete an endpoint directly. Write to support@cloudlytics.com and the support team will disable it from the backend.


 Why do I see "Configured with Cloudlytics" even when I have not configured the bucket?

The message "Configured with Cloudlytics" means that particular bucket or distribution is already configured with Cloudlytics. Check with your admin or team members to verify if anyone else from your organization has signed up for the service.



 How will I be charged and billed for my use of Cloudlytics ?

Cloudlytics has multiple subscription plans based on:

  • Endpoints :

    An Endpoint means a single AWS CloudFront distribution or S3 bucket that you configure with Cloudlytics for log files processing.
    There are multiple plans that allow configuration from one endpoint to unlimited endpoints.
    In Free tier, you can have only one "active" endpoint.

  • Cumulative size of log files processed per month :

    Cloudlytics plans are defined so that you can process up to a pre-defined cumulative size of log files per month for each plan. We aggregate the size of log files by unzipping and uncompressing (in AWS CloudFront) them and process up to that limit, for a particular month. Once you cross the limit assigned to your plan, we send out a notification to the account owner and suspend further processing, until next month or if you upgrade your plan.

  • Retention :

    We store all your log files for 12 months in all plans so that you can have yearly report of your logs. To extend this period you can contact us.

  • Resolution Time :

    Cloudlytics does sampling of log files for reports once in a day, for that entire day.


 What is my billing cycle for Cloudlytics ?

Your billing cycle starts from the day you sign-up for Cloudlytics. Your credit card will be charged at an interval of one month since the day of your sign-up.

You will be charged in advance on the first day of your billing period, for that billing period.


 If I sign up for Cloudlytics in middle of the month, will my logs for the first half of the month be processed ?

Cloudlytics processes log files from a day before your actual signup date. So for example if you sign up on 15th of a month, Cloudlytics will start processing log files from 14th of that month.


 What if I want to process old log files that were generated before I signed up ?

To process your historical log files, you can send a not to support@cloudlytics.com.

Any historical processing will be charged an extra amount depending on the cumulative size of the historical log files. Support team will confirm the amount and raise an invoice before processing the historical log files.



 What types of reports are generated by Cloudlytics ?

Cloudlytics generates multiple types of reports to measure various metrics for access of your assets from S3 or CloudFront.

Reports include, geographical distribution report, IP based access reports, Popular reports and timeline reports among others.


 What is "S3 Stem Browser" ?

S3 stem browser let's you select a stem (a file stored in S3 with complete path) that you want to view report on. It allows you to navigate through the files just like your S3 file explorer on the AWS Console.

All the data of "Keys/Files/Objects" comes from the logs and no data is being accessed from your S3 bucket.


 Geographic Data

Geographic data report provides country wise distribution of your content. You can get a detailed report on each country by clicking on a specific country. Geographic report also providesdetails about data accessed from each edge location.You can filter data using different using different conditions available in the drop downs in the geographic report.

 IP Report

IP report lists all IPs that accessed the content distributed using the endpoint selected over the selected period. You can view different IP access distribution by filtering it browsers and operating systems.

 Timeline Report

Timeline reports provides an overview of requests that were served for a selected period.

 Event Based Report

Event based report gives an overview of how many events were triggered for an endpoint over a selected period. Event reports are only available for CloudFront streaming distributions.

 Popular Reports

Popular reports are a quick way to get the top 10 items for selected elements like stem, browser, IP's or Referrer.

 Log Analysis Status

Log analysis report provides when Cloudlytics processed your data and the amount of data processed.