Whether you’re an Android developer or a release manager, encountering app rejection is a familiar challenge. You may have found yourself pondering the intricacies of Google Play’s review process: How exactly are apps evaluated? Is it a manual review, an automated system, or a combination of both?

With over a decade of experience in the mobile engineering industry, I’ve had the opportunity to work across various domains such as eCommerce, OTT, Gaming, and FinTech. In this article, I’ll share the valuable lessons I’ve learned from numerous app submissions to Google Play, spanning thousands of releases and multiple conversations with the Google Play Experience team throughout my career. These insights are driven by the passion and dedication of the Google Play Policy Experience team, who work tirelessly to understand developers and collaboratively resolve issues to expedite app approvals and support business operations. Additionally, I’ll provide key insights to help you streamline your review cycle and avoid future rejections. Join me on this exciting journey! ☕️🍿

Failure is instructive. The person who really thinks learns quite as much from his failures as from his successes. – John Dewey

Understand and Follow Google’s Policies

App Access

To effectively review your app, Google Play must have unrestricted access to all its components. If any part of your app is restricted, such as sections that require login credentials, you must provide clear instructions on how access can be granted. Historically, because the Play Review team operates globally and can review your app from any country, developers often had to provide exclusive login credentials and expose their infrastructure globally to avoid app rejections. This cumbersome process is no longer necessary.

Sensitive Permissions

For the past five years, I have been immersed in the FinTech industry, navigating the distinctive challenges of developing apps in this domain. Ensuring seamless business operations requires strict adherence to numerous policies and internal audits, encompassing guidelines from the RBI (India’s Apex Bank), Google Play Personal Loan Policies, and others.

If your app requires sensitive permissions like reading SMS, accessing contacts, storage, or location, you might encounter occasional hurdles in the approval process. During these times, the focus shifts to swiftly addressing the feedback and ensuring your app update goes live promptly. You’ll engage in a thorough review of Google’s guidelines, interpret their feedback, and make the necessary adjustments, working diligently to meet their standards and gain approval for your app update.

Curious case of SMS Permissions Group

If your app uses any SMS-related sensitive permissions such as READ_SMS, RECEIVE_SMS, or SEND_SMS, it is crucial to be meticulous in how you convey this information in your permissions declaration form. Clearly articulate the necessity of these permissions and ensure your explanation aligns with Google Play's guidelines. Additionally, cross-verify your use case against the exception use cases provided by Google Play on their policy page to ensure compliance.

Data Safety Declaration

To ensure greater transparency for your app’s users, Google Play requires you to clearly declare what user data your app collects or shares and to highlight your app’s key privacy and security practices. The information you provide in your Data Safety declaration is thoroughly reviewed by Google Play and will be prominently displayed in the Data Safety section of your app’s listing on the Play Store. This helps users make informed decisions about their data privacy and security when using your app.

Financial Services Declaration

Suppose your app’s core functionality involves disbursing personal loans in India. In that case, you must comply with specific requirements and provide supplementary documentation as part of the Financial Features declaration within the Play Console. Upon Google’s request, you must furnish additional information or documents demonstrating your compliance with relevant regulatory and licensing requirements. For more details, please refer to the financial services declaration here.

Useful Tips and Tricks

In addition to the points mentioned above, there are several other key considerations that can help you navigate the challenges of app rejections more effectively. No one wants to see their hard work during development rejected due to policy or compliance issues. However, if it does happen, you can follow the tips below to expedite the approval process and avoid subsequent rejections:

Conclusion

Dealing with app rejections can be challenging, as it directly impacts your business. You want your app up and running as soon as possible, and I hope the insights provided in this article will help you resolve issues more quickly. Google’s commitment to enhancing user privacy and security ensures a safe and trusted experience for everyone. Therefore, it is essential to stay updated with the latest Play Policy Updates, which are available in the Policy Center. I will continue to update this article with new insights and tips as they arise. Until then, keep building 🚀

Do you have additional tips and tricks up your sleeve? Please comment and share your ideas so the entire developer community can benefit and expedite the app approval process. 🤝

For instance, our approach to app development is characterized by innovation and efficiency. Take our flagship LazyPay and Paysense Android apps, for example, both meticulously crafted using Jetpack Compose with Kotlin Coroutines at their core. Similarly, our LazyPay iOS app leverages SwiftUI and Combine to deliver highly responsive user interactions.

But our commitment to excellence doesn’t stop at the front-end. Across both mobile and web applications, we’ve prioritized modularization, laying the groundwork for a scalable architecture that fosters rapid development. By embracing modular frameworks, libraries, micro-apps, and micro-frontends, our team has not only streamlined development processes but also significantly reduced time-to-market for new features and enhancements.

Behind the scenes, our Mobile Backend Service, aptly named Sauron, forms the backbone of our app ecosystem. With a focus on performance, scalability, and availability, the team ensures that Sauron operates at peak efficiency to support our applications’ demands. My interest in Kubernetes(K8s) stemmed from a direct involvement in optimizing Sauron’s infrastructure costs, a journey that has further deepened my appreciation for scalable and cost-effective solutions.

As I delve deeper into the actual problem and its solution, I will also go through the journey of why and how I earned the CKA and CKAD K8s certifications in detail. This journey reflects my ongoing commitment to mastering new technologies and driving continuous improvement. It’s a testament to the belief that growth lies beyond our comfort zones, and I strive to inspire others to embrace this philosophy fearlessly.

The more that you read, the more things you will know. The more that you learn, the more places you will go. — Dr Seuss

The Trigger: Sauron Excessive Logging Issue 📈

Our Mobile Backend Service, Sauron, functions as a Backend-for-Frontend (BFF) layer, orchestrating interactions with our internal microservices. While devoid of any business logic, Sauron optimizes caching and transforms service responses to align with client requirements. Despite its pivotal role, Sauron was burdened with a substantial log ingestion volume, consuming approximately 600GB of logs daily, which surged to a staggering 1.1TB during peak repayment cycles.

To manage this influx, we had initially over-provisioned infrastructure resources, including production clusters and Elasticsearch, to accommodate the heightened demand. However, the escalating costs prompted us to reassess our approach. Recognizing the need to curb infrastructure expenses, our team embarked on a mission to reduce log ingestion.

Reducing the log ingestion by ~90% 🚀

Our efforts to optimize log ingestion yielded substantial benefits, leading to significant cost savings through infrastructure scaling. We implemented several key initiatives to achieve these results:

1. Eliminated aspect logging from API calls to reduce unnecessary verbosity.

2. Trimmed down cache hit-and-miss logs, focusing only on essential data.

3. Reduced excessive response logging to minimize log size.

4. Implemented smarter logging levels (Verbose, Debug, Warning, Error) to prioritize critical information.

5. Identified and removed redundant API calls and their corresponding logs.

6. Utilized local in-memory cache to reduce the need for multiple internal service calls.

7. Established a meticulous and automated code review process to identify and address logging issues proactively.

8. Tweaked the Filebeat configuration using drop fields.

These initiatives were implemented gradually and methodically, ensuring minimal disruption to production environments and preserving our debugging capabilities. Over the past six months, our efforts have led to a remarkable reduction in log size, reaching 50GB and 100GB on regular and repayment cycle days, respectively. Moreover, we achieved a 60% reduction in infrastructure usage, resulting in substantial cost savings for the organization.

Unforeseen Downtime: Navigating Challenges in Optimization ⚙️

However, our journey wasn’t without its challenges. Despite our best efforts, one particular initiative led to an unexpected hiccup. Following the implementation of various optimizations, we conducted a scale-down activity to assess performance and scalability. Unfortunately, this decision resulted in an unforeseen production downtime for a couple of hours.

As we investigated the root cause, we discovered that our assumptions about the behaviour of the Horizontal Pod Autoscaler(HPA) were flawed. The HPA failed to trigger auto-scaling due to preset limits on the number of requests each POD could manage. Despite the performance metrics like CPU and memory remaining within acceptable thresholds, the POD couldn’t process the incoming requests as expected. This revelation was a pivotal ‘Aha!’ moment for me, prompting a deep dive into Kubernetes and a pursuit of relevant certifications.

We discovered that the HPA wasn’t equipped to meet our specific needs and wasn’t suitable for our use cases. We needed triggers beyond standard performance metrics, such as the number of requests or queue lengths, to effectively auto-scale our deployments. To prevent similar incidents in the future, we implemented Keda, an event-driven auto scaler capable of dynamically adjusting scaling based on multiple factors and attributes.

Why Kubernetes Matters: A Personal Journey 💡

• Cloud-native apps: As mobile apps increasingly rely on cloud-based services, understanding K8s can help me to design and deploy cloud-native applications that scale and perform well.

• Backend Service: Since we faced an issue during one of the scale-down activities of our Sauron service earlier, knowing Kubernetes from the ground zero would help me to communicate effectively to our backend team in optimising the services further.

• Containerization: K8s is built around containerisation(e.g. Docker) and that’s why understanding it better can allow to package the app’s code & dependencies into a single light-weight container, making it easier to manage and deploy.

• Micro-services architecture: K8s is well-suited for the microservices architecture which is increasingly popular in the front-facing apps. Knowing it better can help in designing and deploying the scalable distributed systems that consist of multiple services communicating with each other.

• DevOps collaboration: K8s is a key tool for DevOps teams. By learning it, I can now collaborate more effectively with my DevOps colleagues and improve the overall efficiency of our team.

• Career Growth: Knowing K8s can open up new learning and career opportunities in cloud computing, DevOps and Distributed systems. While I don’t aim to pursue my career in DevOps, having an advanced knowledge around the infra-structure and its internal workings can open doors to many untouched realms of engineering.

• Better communication with Stakeholders: With K8s knowledge, I can collaborate more effectively with the stakeholders, including product managers, customers and executives about the technical aspects of the backend service. I can now debate around the infra capabilities and its concepts in a more clear and concise manner.

• Improved debugging skills: Understanding K8s can help in debugging production issues much faster and more efficiently as I will have a deeper understanding of the underlying infrastructure and services. This can in-turn help in improving the overall quality and reliability of our mobile and web apps.

• Future-proofing: K8s is a rapidly evolving field, and learning it can help me future-proof my skills and stay ahead of the curve in the industry. Having expertise on the mobile engineering aided with K8s knowledge can help me in taking advantage of new technologies and trends as they emerge.

My Certification Journey 🎓

Learning Kubernetes isn’t just a journey — it’s an adventure filled with challenges, discoveries, and moments of triumph. As someone entrenched in the world of experience engineering, diving into Kubernetes meant immersing myself in every layer of its infrastructure. From the fundamental building blocks to the intricate inner workings, I embarked on a journey to unravel the mysteries of Kubernetes and unlock its full potential.

But learning Kubernetes wasn’t just about acquiring knowledge — it was about pushing boundaries and expanding horizons. As I delved deeper into the intricacies of container orchestration, I realized the immense value of formalizing my expertise through certification. It wasn’t just about validating my skills — it was about embracing a new chapter in my journey as a technologist, driven by a passion for continuous learning and growth.

Road to CKAD: Certified Kubernetes Application Developer

• Completed Docker for the Absolute Beginner course, gaining foundational knowledge in containerization.

• Completed Kubernetes for the Absolute Beginner course, acquiring a comprehensive understanding of Kubernetes fundamentals.

• Successfully completed the Certified Kubernetes Application Developer (CKAD) course, including extensive practice tests and Mock Exams to solidify skills.

• Engaged in hands-on practice with VIM, mastering essential shortcuts crucial for efficient coding and exam success.

• Completed CKAD exercises by Dimitris-Ilias Gkanatsios and Bhargav Bachina, reinforcing practical skills and exam readiness.

• Practiced Killercoda and KillerShell scenarios for CKAD, simulating real-world exam conditions and honing problem-solving abilities.

• Learned HELM to effectively deploy and manage Kubernetes applications, further expanding proficiency in container orchestration.

I dedicated my off-hours and any available leisure time outside of my regular work schedule to diligent practice. This commitment bore fruit when I successfully passed the CKAD certification exam on my first attempt, achieving an impressive score of 88 out of 100. 🥳

Road to CKA: Certified Kubernetes Administrator

Achieving my CKAD certification was a moment of genuine satisfaction, considering I ventured into a domain completely unfamiliar to me. As I delved deeper into Kubernetes concepts, my interest in mastering the intricacies of Kubernetes intensified. This curiosity led me to set my sights on obtaining the CKA: Certified Kubernetes Administrator Certification, a challenging yet rewarding pursuit.

• Successfully completed the Certified Kubernetes Administrator (CKA) course and its associated lab with Mock Exams on KodeKloud.

• Learnt K8s the hard-way by Kelsey Hightower.

Transitioning from CKAD to CKA was a natural progression, considering my solid foundation in Kubernetes Application Development. The CKA exam, with its emphasis on both administrative and development aspects of Kubernetes, posed a new challenge. This certification requires a comprehensive understanding of security, roles, cluster debugging, high availability, networking, and more. After diligently preparing for several months following my CKAD certification, I undertook the CKA exam and was thrilled to achieve a score of 99 out of 100, exceeding even my own expectations. 🥳

Unlocking New Horizons: Embracing the Kubernetes Journey 🌱

As I reflect on my journey from being a mobile engineer to earning CKA and CKAD certifications, I realize that Kubernetes has become an integral part of my skillset. The knowledge and experience I gained have not only helped me in my current role but have also opened up new opportunities for growth and collaboration. If you’re a developer irrespective of your domain expertise and looking to expand your horizons, I encourage you to embark on the Kubernetes journey. It may seem daunting at first, but with persistence and dedication, you’ll find that the benefits far outweigh the challenges. Here are some key takeaways from my experience:

• Kubernetes is not just for ops teams; it’s for anyone who wants to build scalable, reliable, and efficient systems.

• Learning Kubernetes requires practice, patience, and persistence.

• The Kubernetes community is vast and supportive; don’t be afraid to ask questions or seek help.

• Kubernetes is a constantly evolving field; stay curious and keep learning.

My journey with Kubernetes has been a transformative experience that has helped me grow both professionally and personally. I hope that my story will inspire you to take the leap and join the Kubernetes community. Embrace the journey, and get ready to unlock new possibilities in your career! 🚀

During the recent LazyPay app overhaul, we strategically re-architected our API design, adopting a hybrid approach that seamlessly blends data and user interface elements. This shift enhances scalability and fosters extensibility, ensuring our app remains agile and adaptable to evolving user needs.

Our dedicated team meticulously crafted the LazyPay Android and iOS apps, leveraging the latest technology stacks to deliver a seamless and immersive user experience. Check them out today on the respective app stores and share your valuable feedback! ❤️

Challenges

When an idea is born, the app around it is built in the fastest and crudest way possible. Like other startups, LazyPay apps were developed in a similar manner. The media like images, videos, and animations(Lottie) being served on the apps were stored on AWS S3 which were consumed by apps directly through the S3 URLs. The brute-force approach obviously had problems at many levels.

• Performance Bottlenecks: Although we backed our S3 buckets with AWS CloudFront for utilizing caching and security gains, the apps were getting media of the same sizes that were stored by us leading to higher bandwidth consumption.

• Screen Size and Rendering Issues: To be highly responsive, the APIs should serve the response with millisecond latency. Unfortunately, this is not the case most of the time, especially over media. This becomes more painful when a user on a mobile device requests for an asset and an unwanted cache-miss leads to the slow screen rendering making the app unusable.

• Page Load Time: Since the media being served was not optimized, it directly affected our page load time leading to delayed interaction by the user with our product offerings. This became even worse when our users were connected to a 4G network and residing in any low connectivity zone or even roaming.

• Heavy Maintenance: To support multiple clients and optimize asset delivery, we were uploading multiple variants of the same image based on the platform(android/iOS/web) and device/browser density. For ex: If we had a banner for promoting Xpress Loan on the app, we would be uploading 7 variants of this image ranging from mdpi to xxxhdpi on Android and 1x to 3x on iOS respectively.

• Unstructured asset management: Since every client was handling the assets according to their own needs, there was no order to the files that were being uploaded, creating a mess of buckets and folders everywhere.

Cloudinary as the Media Optimizer

AWS S3 for cloud storage of assets is the right choice and we are sticking to it. However, we have laid out a consistent process around asset management and brought a true order to it. Also, to reduce the network footprint of our apps and enhance their performance, we have invested a lot in adopting Cloudinary across all of our clients.

Cloudinary is a powerful media hosting and optimization platform that delivers high-quality media without much maintenance effort.

From S3 via Cloudinary

Connecting S3 to Cloudinary is pretty straightforward.

1. CreateMedia Source on Cloudinary for the S3 bucket

2. Create an Optimization Profileon Cloudinary and attach the media source with it.

3. Provide the base transformation on the profile as one of the custom transformations.

Ease of Maintenance

The best part of using Cloudinary is its zero maintenance effort. Recall, when we said earlier that we were uploading 7 variants of an image to support multiple clients. Not anymore! With Cloudinary, we are now uploading only one master image which serves all our clients without compromising on the app performance.

Cloudinary Transformations

Cloudinary can serve different variants of the master image on the fly via transformations. We have created named transformations for all of our clients to not only serve the optimized asset according to the device/browser but also shorten the image URLs. These transformations have helped significantly in lowering our maintenance efforts and increasing our productivity.

• On Android, we have 5 named transformations mapping with mdpi, hdpi, xhdpi, xxhdpi, xxxhdpi created based on display metrics.

• On iOS, we have 3 named transformations mapping with 1x, 2x and 3x created based on the scale factor.

• On the web, we have 3 named transformations mapping with small, medium and large created based on aspect ratio.

LazyPay android app requests an image on a medium display density device -> Android app appends the transformation(tx=t_droid_xhdpi) dynamically on the requested Cloudinary image URL -> Cloudinary checks for the transformed image in cache -> If cache hits, serve the image, else transform the image from the master one on the fly, cache, and then serve.

Optimizations

Cloudinary’s out-of-the-box image optimizations helped in further reducing data consumption.

• An image can be served in any format webp, avif or png automatically by Cloudinary optimized based on clients (Android/iOS/Web).

• Cloudinary’s multi-CDN helps in delivering the images faster and more reliably.

• Bytes saved are bytes gained for processing other high-intensive APIs within the app. Using Cloudinary helped us to save 70% of the bytes when compared to the originals. For eg: An image of 100 KB on S3 required only 30 KB to be served via Cloudinary.

• Apart from the byte savings, we even monitored the performance of the assets being served through Cloudinary and compared them with AWS S3 through Firebase Performance Monitoring. Monitoring insights suggested that image URLs served through Cloudinary were performing faster by 20% as compared to S3.

Go-to-market through controlled feature rollout 🚀

Migration to Cloudinary was a big change for us and we wanted to tread very carefully before adopting it on all platforms. Launching in one shot might have given an experience where the user would not be seeing any asset on our app giving the worst user experience. Reasons could be many: Dirty Setup, Missing Media, Heavy Loads, etc. That’s why, we controlled the feature and rolled it out incrementally through Firebase Remote Config. Cloudinary’s error reporting facilitated the process by properly displaying the asset URLs giving HTTP 404s & 400s. We kept on stabilizing the error count and increased the rollout only when the error graph had flattened out. While doing so, we also uncovered some new and age-old issues.

• HTTP 404 errors received on Cloudinary were for the assets that were never uploaded and missing on S3. This is a typical case of a hibernating bug finally waking up. We had to find those assets somehow, upload them on S3, and fix the errors.

• Since we added a blanket transformation across all of our assets uploaded on S3, this impacted assets like gif or pdf giving HTTP 400 error due to increased MegaPixel count as we were asking for better quality gifs on high-density devices. Error from Cloudinary: Maximum image size is 8 Megapixels. Requested 9.72 Megapixels
  We then added a filter to exclude the media types like gif and pdf from being transformed.

Wrap up

We are already using Cloudinary for serving assets on all of our mobile apps and slowly rolling it out to our web apps.

FinTech has many unique challenges and as we are progressing, we are sharpening our skills, re-architecting the apps, and solidifying our infrastructure to build for India scale. We are challenging the status quo at each moment all aimed at giving an awesome user experience. Stay tuned for the next one!

• How many times have you had to build and install another build to test a functionality pointing to different environments?

• How many times have your Backend engineers wanted to test the functionality during the development on their local setup?

On numerous counts. Isn’t it? On every occasion, you had to start the tedious build process which will take its own sweet build time thus delaying the quality and testing turnaround time.

Not anymore! At LazyPay, we developed an in-app flow to let anyone switch their environment without requiring new builds enhancing our developer and QA productivity. This article will help you understand the path we took and guide you accordingly in creating One Build to Rule Them All for your Android apps. 🍿 ☕️

Conventional Process ✍️

In general, to support different environments, we will have a 1:1 mapping of productFlavor the environment’s host URL. For the sake of consistency and to extract the best of Kotlin, we will be talking in terms of Gradle’s Kotlin DSL for all build-related scripts. For eg. Say, our app should support two backend environments Production and Staging. This is what our build.gradle.kts looks like:

android {
  ...
  flavorDimensions.addAll(listOf("env"))
  productFlavors {
    create("prod") {
        dimension = "env"
        buildConfigField("String", "HOST_URL", "https://prod.api.com")
        buildConfigField("String", "SOME_KEY", "prod_key") 
   }
    create("staging") {
        dimension = "env"
        buildConfigField("String", "HOST_URL", "https://staging.api.com")
        buildConfigField("String", "SOME_KEY", "staging_key")     
    }
  }
  ...
}

When we create prodDebug and stagingDebug apps, each will be pointing to their respective environment's HOST_URL. To test a feature’s behaviour on multiple environments, we will build separate apps accordingly.

The Idea💡

On Android, when we insert any key-value pair under buildConfigField and compile the source, BuildConfig.java is auto-generated and this changes according to our buildFlavor defined values. For the example above, BuildConfig.java will look something like this for stagingDebugbuild variant:

public final class BuildConfig {
public static final boolean DEBUG = Boolean.parseBoolean("true");
public static final String APPLICATION_ID = "com.example.app";
public static final String HOST_URL="https://staging.api.com";
public static final String SOME_KEY = "staging_key";
}

If we can somehow tweak this BuildConfig.java to accommodate the configurations for all supporting buildFlavors and environments, this will solve the problem for us. Unfortunately, BuildConfig.java is generated at compile time and cannot be dynamically changed once the build process has been completed. What if we don’t have to change this dynamically and still be able to hold the configurations for different productFlavor. Let’s build on top of this idea.

Injecting build configuration 🏗

We now understand that BuildConfig.java holds all the values according to the key type defined under buildConfigField. For String key as HOST_URL, it generated public static final String HOST_URL with its subsequent value. To allow for switching environments from one build, we would like to have all the configurations(prod/staging, etc.) available under our BuildConfig.java.

Mapping Build Flavor with Configurations

We will set up all the buildFlavors and their corresponding environment configurations in a way such that the auto-generatedBuildConfig.javacontains the following:

public static final java.util.Map<String,String> PROD_MAP = 
new java.util.HashMap() {{put("HOST_URL","https://prod.api.com"); put("SOME_KEY","prod_key")}};

public static final java.util.Map<String,String> STAGING_MAP = 
new java.util.HashMap() {{put("HOST_URL","https://staging.api.com"); put("SOME_KEY","staging_key")}};

public static final java.util.ArrayList SET_OF_FLAVORS = 
new java.util.ArrayList() {{add("prod");add("staging");}};

If we can generate the PROD_MAP and STAGING_MAPalong with a set of supported flavors, our app will have all the information available to switch the configuration at runtime. The concept is also driven by a standard code generator like KotlinPoet.

Before we do this step, we should also do housekeeping of our build.gradle.kts to move all the build configurations to one place for easy maintenance. We will create Config.kt inside the buildSrc folder for the same.

Once we have segregated the build configurations, we will now tweak our build.gradle.kts file to auto-generate the subsequent flavor map and set on BuildConfig.java.

• Under defaultConfig, we will ensure that all our Keys are available on BuildConfig file.

Config.keyList.forEach { key ->
buildConfigField(“String”, “KEY_$key”, “\”$key\””) 
}
// This generates the keys and writes on BuildConfig
public class BuildConfig {
  public static final String KEY_HOST_URL = "HOST_URL";
  public static final String KEY_SOME_KEY = "SOME_KEY";
}

• Next, to have the configurations available for each productFlavor, we will change our build.gradle.kts to generate <flavor>_MAP on BuildConfig.java.

applicationVariants.all {
 defaultFlavors.forEach { value -> 
  buildConfigField(
  "java.util.Map<String,String>",  // type 
  "${value.toUpperCase()}_MAP", // name
  variantFields(value)      // value          
  )            
 }
}
fun variantFields(flavor: String): String {
    val fields = variantFieldMap[flavor] // Defined in Config.kt
    val fieldsBuilder = StringBuilder("")
    fields!!.forEach { entry ->
   fieldsBuilder.append("put(\"${entry.key}\",\"${entry.value}\");")
    }
    return "new java.util.HashMap() {{$fieldsBuilder}}"
}

Since buildConfigField) only takes String as the value type that can be written on BuildConfig, we have provided the typeexplicitly as java.util.Map<String, String> and the value as the build configurations map.

This will write PROD_MAP and STAGING_MAP on BuildConfig like this:

public static final java.util.Map<String,String> PROD_MAP = 
new java.util.HashMap() {{put("HOST_URL","https://prod.api.com"); put("SOME_KEY","prod_key")}};
public static final java.util.Map<String,String> STAGING_MAP = 
new java.util.HashMap() {{put("HOST_URL","https://staging.api.com"); put("SOME_KEY","staging_key")}};

• Lastly, we will also write the productFlavors on BuildConfig which app will support. This will help in showing the options to the end-user in switching the environment.

applicationVariants.all {
 buildConfigField("java.util.ArrayList<String>", "SET_OF_FLAVORS", getFlavorList())
}
fun getFlavorList(): String {
    val flavorBuilder = StringBuilder()
    defaultFlavors.forEach {
        flavorBuilder.append("add(\"$it\");")
    }
    return "new java.util.ArrayList() {{$flavorBuilder}}"
}

This will write the list of flavors on BuildConfig.

public static final java.util.ArrayList<String> SET_OF_FLAVORS = 
new java.util.ArrayList() {{add("prod");add("staging");}};

Once we have executed the above steps, our apps build.gradle.kts should look something like this:

To ensure the security of our app, for any buildType of release variant like prodRelease, we will not write other flavor configurations on BuildConfig and just create the RELEASE_MAP (line:13 on above GIST)

Accessing the BuildConfig 🔓

We have everything written on BuildConfig as required to develop the flow for dynamically switching environments. Let’s connect the final dots.

We will create an abstraction over BuildConfig for accessing their values. If you are using Dependency Injection via Dagger or HILT, it will help in keeping your concrete classes clean enhancing the Unit Testing. Here’s what our BuildConfigProvider will look like this:

BuildConfigProviderImpl will have the detailed implementation of all the APIs exposed via BuildConfigProvider.

Now, all we will need to do is to define a binding for BuildConfigProvider such that Dagger or HILT knows how to provide the instance of this abstraction. Once this is set up, we will be able to inject BuildConfigProvider as a dependency wherever required.

@Module
@InstallIn(SingletonComponent::class)
abstract class AppBindingModule {
 @Binds
 abstract fun provideBuildConfigProvider(buildConfigProvider: BuildConfigProviderImpl): BuildConfigProvider
}

Change Environment w.r.t Flavor 🚀

With the above implementation in place, we were able to provide the environment-switching capability right from within the LazyPay app.

LazyPay debug app showcasing supported environments. Build using Jetpack Compose ❤️

Add-On: Custom Domain Support 🚀 💯

While we have developed the environment switching feature, there is one feature missing in our implementation and that is to support custom domain configuration. This also came as huge demand from our Backend and BFF Developers as they wanted to dev test their APIs from their local setup. Keep reading to know how we built it.

Until now, we have provided support for all the environments where we have 1:1 mapping of productFlavor and its build environment(HOST_URL). We will now expand our idea to provide custom domain support.

• Since only HOST_URL will have to change, we will fix the productFlavor where other configurations remain the same and only HOST_URL can be edited. We will be configuring it on our staging flavor.

• We will tweak our BuildConfigProvider to expose the setter/getter for changing the HOST_URL.

    class BuildConfigProviderImpl : BuildConfigProvider {
        ...
     override fun setHostUrl(baseUrl: String) {
      // store in SharedPreferences
     }
     override fun getHostUrl(): String {
         val baseUrl = // get from SharedPreferences else return empty
         return baseUrl ?: getValue(BuildConfig.KEY_CS_JAVA_URL)
     }
    }
  

• Unfortunately, the app will start throwing the below error when the domain is changed.

    java.net.UnknownServiceException: CLEARTEXT communication to <custom_domain> not permitted by network security policy
  

• Solving java.net.UnknownServiceException: As all our productFlavor with the HOST_URL like https://prod.api.comwill go via secure connections, [clearTextTrafficPermitted](https://developer.android.com/training/articles/security-config#CleartextTrafficPermitted) is always set to false by default. This will avoid fraudulent and insecure connections to our app. But, custom domain setup will require us to tweak the [network-security-config](https://developer.android.com/training/articles/security-config) to allow for insecure connections. We will create separate network-security-config files for different buildType. Our debug variants will allow for [clearTextTrafficPermitted](https://developer.android.com/training/articles/security-config#CleartextTrafficPermitted) while the release won’t.

    // Create debug folder and place network-security-config under there
    <network-security-config>
        <base-config cleartextTrafficPermitted="true"/>
        <debug-overrides>
            <trust-anchors>
                <certificates src="system" />
                <certificates src="user" />
            </trust-anchors>
        </debug-overrides>
    </network-security-config>
  

At LazyPay, we provided custom domain support on our sbox environment like this.

LazyPay debug app showcasing custom domain support. Build using Jetpack Compose ❤️

Once a user attempts to switch the domain/environment on app, it is recommended to clear any app cache restarting the app as a fresh launch.

Add-On: Obfuscation on Release Variant 🚀 💯

If we like to have the above feature built on a variant where obfuscation and minification run via Proguard typically release variant, our apps will crash due to code obfuscation. To fix this, we will add the following rule on proguard-rules.pro:

# Not obfuscating config-maps of BuildConfig as we are accessing them via reflections
-keepclassmembers class com.citrus.citruspay.BuildConfig {
    public static final java.util.Map PROD_MAP;
    public static final java.util.Map STAGING_MAP;
}

Conclusion ✅

We have provided all our learnings in building a feature that directly enhances the developer's productivity and experience. Time saved in generating different build variants is time gained for faster development and testing cycles.

As we gear up for a new age in financial technology, we are also striving hard to prepare our apps built for India scale following Modern Android Development, writing UI in Jetpack Compose, Unit Tests, and much more. At LazyPay, we are working every day to improve not only our products but also excel in terms of how we engineer them for scale. If this excites you, come join our Experience Engineering team at PayU Credit. We are hiring across all domains.

PS: Next blog will talk about how our iOS engineering team developed a similar feature to improve their productivity. Stay tuned. 🙌

CI/CD bridges the gaps between development and operation activities and teams by enforcing automation in the building, testing and deployment of applications. — Wikipedia

CI/CD is not new, and it has been at the forefront of adopting good software engineering practices. While it is vital for all, it becomes even more critical for companies that are just getting started, typically startups. They follow the infamous quote of moving fast and breaking things. When the system begins becoming unstable, the need for proper automation(development/testing/deployment) kicks in while still maintaining development and release speeds. It eventually boils down to adopting a proper CI/CD pipeline right from the beginning and improving upon it as the underlying services/apps scale. This article provides an overview of the core concepts for building a CI/CD pipeline for Android via Fastlane and GitHub Actions.

Sit back and onboard the build automation rocket 🚀

Building CI/CD pipeline 🔨

You can skip this section if you do not have any private libraries.

Preface: Migration from JFrog’s Artifactory to GitHub Packages

If your Android app relies on several in-house libraries, you would be probably using JFrog’s Artifactory to manage your repositories served via Maven. While Artifactory is incredible in its own way, you lose out on the principle of a single source of truth. That typically means that if your code repository is on GitHub, you will ideally want all the related packages also available in the same place. This helps in better maintenance and visibility. When you start setting up a CI/CD pipeline on GitHub action, you will face the first blocker as the packages are hosted privately on Artifactory via intranet requiring a VPN connection. Here are some out-of-the-box solutions:

1. Go Public  -  You can make Artifactory publicly accessible, obviously under the authenticated environment. If the packages are private, why would you want them to be hosted publicly? Also, isn’t it still defeating the idea of a single source of truth?

2. Use Self-Hosted Runners -  Github also provides the power to use self-hosted runners to customize the environment according to your workflow needs. Since the environment will be hosted and created according to your needs, you can easily pull out the private libraries while building the app on Github action. This sounds good but fails due to the massive efforts it would require to build and stabilize while still investing time in the maintenance of it.

“We can not solve our problems with the same level of thinking that created them”
― Albert Einstein

As mentioned earlier, the aim of the pipeline is to have a cleaner separation of concern with clearer visibility having GitHub as the single source of truth (code, wiki, versioning, changelog, actions, releases, etc.) for your repositories. Proper version/release management, faster deployment, and deeper integration with GitHub services will help in running a smoother pipeline and enhance it for future needs.

If you are already publishing your libraries to Artifactory via Maven, it would require little effort in build.gradle file to start publishing .aar files on GitHub. You can maintain agithub.properties that contains github_token and github_username to give the capability to publish locally for fallback. The pom file for the .aar packages will also be published under GitHub packages.

Using ./gradlew publish locally, you can test your integration and find that the libraries are properly getting published on GitHub packages under your targeted repository.

Minimal changes are required in your Android project’s build.gradle file to consume the library.

Github Action and Fastlane working together

Once you have the in-house libraries being pulled in via GitHub, you can start creating the workflow for your Android app on GitHub actions. You can have two different strategies depending upon pushes on the target branches(development & master) These are aimed at not only removing the manual intervention altogether by having continuous integration embedded deep inside the workflow but also focussing on continuous deployment by automating the overall release process right from building the bundle and uploading to the Play Store. Let's discuss the important steps in the workflow:

• No hardcoding  - Every variable used in the workflow, is made to come via GitHub secrets. For creating a release build, you can first encrypt your .jks and release.properties files containing signing information and then store them on GitHub secrets.

    // Encryption
    gpg --armor --symmetric release.properties
    // Decryption 
    gpg -d --passphrase "<Password used for encryption stored as a GitHub secret>" --batch release.properties.asc
  

• Similarly, to upload your release on the Play Store, you can encrypt your Google service account file and then save it as a secret.

    # Decrypt release keystore and release properties file from github secrets
    - name: Decrypt files
      id: decrypt_files
      run: |
      # Creating encrypted keystore file from secret saved on GitHub
        echo "${{ secrets.YOUR_APP_RELEASE_STORE }}" > customer.jks.asc
      # Decrypting keystore from passphrase(saved as secret) used earlier for encryption
        gpg -d --passphrase "${{ secrets.YOUR_APP_RELEASE_PASSWORD }}" --batch customer.jks.asc > config/customer.jks
      # Creating encrypted release.properties file from saved secret.
        echo "${{ secrets.YOUR_APP_RELEASE_PROPERTIES}}" > release.properties.asc
      # Decrypting release.properties.asc with passphrase(saved as secret) used earlier for encryption
        gpg -d --passphrase "${{ secrets.YOUR_APP_RELEASE_PASSWORD }}" --batch release.properties.asc > config/release.properties
      # Creating encrypted play service account file from saved secret.
        echo "${{ secrets.YOUR_PLAY_RELEASE_SERVICE_ACCOUNT}}" > <your_play_service_account_filename>.json.asc
      # Decrypting play service account with passphrase(saved as secret) used earlier for encryption
        gpg -d --passphrase "${{ secrets.YOUR_APP_RELEASE_PASSWORD }}" --batch <your_play_service_account_filename>.json.asc > <your_play_service_account_filename>.json
      # Creating tester-groups file used for publishing on firebase app distribution
        echo "android,qa" > tester-groups.txt
      # Creating empty github.properties to avoid the build failure since our local builds depend on it
        echo " " > github.properties
  

• Setting up Fastlane and Firebase tools — If your app is already using Fastlane, you can run Fastlane commands manually to trigger a build and send it on Slack and Firebase App distribution. It is about auto-triggering those commands from your workflow and the rest will be taken care of by Fastlane itself. Slack is used to communicate the status of our build workflow and send .apkor .aab to keep the concerned folks notified in real-time by using different GitHub Actions described further in the article.

Firebase App Distribution enables you to instantly manage and deliver pre-release versions of your app with trusted testers in order to get feedback and find issues ahead of releasing to production.
You can now distribute android app bundle too from Firebase App Distribution 🎉

• Faster workflow  - Set up Gradle, Download the dependencies, Set up Ruby, Install Firebase Tools, Set up Fastlane, Install Bundler, etc are the steps that get executed for every workflow trigger. This will affect your GitHub Actions in billing minutes leading to higher payments. You can use [actions/cache@v2](https://github.com/actions/cache) to have a faster workflow execution time.

  %[https://gist.github.com/reactivedroid/c7e3bf75da50a5b532439c5bbc64f1b3]

You can use Gradle cache and Gems cache to avoid downloading the jars and gems if there is no change in the dependencies. It is important to create a cache key in such a way that it can be invalidated when required thus avoiding stale builds. For Gradle cache, checksum acts as the cache key which is calculated by running the checksum script( [checksum.sh](https://github.com/chrisbanes/tivi/blob/main/checksum.sh))[Thanks to Chris Banes]. For gems, the hashFiles(Gemfile.lock) acts as the cache key. Once there is a Gradle/Gems cache hit, the setup time will drastically reduce giving faster workflow runs.

• Running UI/Unit tests — UI/Unit tests play a vital role in making the app robust. It helps to track any broken features upfront when a huge refactoring/re-architecture task takes place. If the Unit/UI tests are not working as expected, you can fail the CI/CD pipeline to keep the tests always up-to-date with the newer changes. Once the tests have been executed successfully, you can upload the test reports and results on the workflow for future analysis.

  %[https://gist.github.com/reactivedroid/abf7f60c34c5f821df9658d55543cff1]

• Creating release builds— Once you have made the necessary setup to run Fastlane via bundle, you can execute lane based on the branch push type development& master to create debug and release builds respectively. You can club all the steps under one workflow separated if: github.ref == ‘refs/heads/master’to have your development and release builds go seamlessly. Let’s break this down even further.

1. Distributing app bundle via Fastlane - You can write a lane which will bundleProdRelease and then push the release tag on themaster branch. This will then publish the build on the Slack channel and Firebase App Distribution. This is how the lane looks like in fastfile

   %[https://gist.github.com/reactivedroid/0dbbe11ebec06a2c671543dc78e19d31]

2. Lane Options on Fastlane - Fastlane needs some values to run the distribute_bundle lane. You can store the required values as secrets on GitHub and then inject them into Fastlane as lane options to be used later.

   %[https://gist.github.com/reactivedroid/c8c513a5643a24cdd508482b798d186e]

3. Creating GitHub release from Workflow  - Once the release tag is pushed on Git via build_prod_release step above, you can save it as tag_name and use it to create our GitHub release. To know what actually went in the release, you can check-in changelog.mdor release_notes.txt into version control. This file will be used to push releases both on Firebase app distribution and GitHub. You can also use the checked-in release notes to create your GitHub releases right from workflow while uploading other assets like bundle/apk.

   %[https://gist.github.com/reactivedroid/cc1c5e7554c61a49c312fc2e0a3c1e5f]

4. Uploading to the Play Store  - The final piece is automating the deployment process from the workflow. You can create a service account file from Google Cloud Console, associate it with Play Store by giving necessary permissions and use it in your workflow to upload on internal track. Once the build is approved by the Play folks, you can promote it to production.

   %[https://gist.github.com/reactivedroid/0b238fd16e5da39640f02fd448b352e0]

5. Uploading build outputs and send status  - Hope you are still with us :) The final step includes uploading all the build outputs on workflow and then notifying about its status on Slack via action.

   %[https://gist.github.com/reactivedroid/6f7f04470bd67a6e8624ebcb6c94539d]

Phew! That was a long read, right 😌. I understand that creating a seamless CI/CD workflow for Android builds can be tricky and it becomes even more complicated when you want to automate each and every step from continuous development to production.

For that reason, I have tried my best to provide you with as many details as possible so that setting up CI/CD pipeline should be a fairly easy task going forward and you can reap its benefits in the longer run. Obviously, this smooth setup would not have been viable without the ever-growing open-source contributions ❤️. You can use slack action to get the status of your workflow right into your Slack channel. Sweet, isn’t it!

Conclusion

This is not the only way to create a CI/CD workflow on GitHub action and I am all ears to listen to your creative ideas. I will feel more happy and content if this article helps you in any way possible. Till then, keep learning and sharing. #BetterTogether

Face Detection via MLKit

When we started on our journey to improve the Selfie experience, the aim was very simple: Our customers should be able to capture their Selfies clearly for faster loan processing. So, we utilised the power of machine-learning mainly Face detection MLKit(earlier via Firebase) to solve just that. We went with the unbundled face model approach for MLKit as the bundled one would increase the app size by 16MB. It pulls in the face models lazily when the app is being initialized and opened for the first time.

// Add this in your AndroidManifest.xml to automatically download the face model from the Play Store after the app is installed
<meta-data
android:name="com.google.mlkit.vision.DEPENDENCIES"
android:value="face" />

Setting up Camera 📸

The choice of camera library decides the efforts of implementation. Until CameraX becomes stable, we would want to use the library that provides easy-to-use APIs that are compatible with different Android OS, and then we found CameraView. CameraView is well-documented and powered by Camera1(<API 21) and Camera2(≥API 21) engines making it fast & reliable. It has all the features, we need, the most important being Frame Processing Support. It also offers support on LifeCycleOwner where the CameraView handles the LifeCycle event on its own, mainly asking for permission in onResume, cleaning frame processors & listeners, and destroying cameraViewin onDestroy of fragment/activity. By default, CameraView offloads the Frame processing to the background thread so that these frames can then be consumed by the face detector synchronously.

cameraView.apply {
    setLifecycleOwner(viewLifecycleOwner)
    facing = Facing.FRONT
    addCameraListener(cameraListener())
    addFrameProcessor {
        faceTrackerViewModel.processCameraFrame(it)
        cameraOverlay.setCameraInfo(
            it.size.height,
            it.size.width,
            Facing.FRONT
        )
    }
}

It is important that you set the width and height of cameraOverlay to the received frame’s width and height. While testing initially on different devices, the face detected rectangle went away from where the face actually resides in. More details are present in the issue and how we fixed it.

Reactive Frame Processing for Face Detection 📽️

The frame is then sent to the frame processor via faceTrackerViewModel to detect a face in it.

fun processCameraFrame(it: Frame) {
    val byteBuffer = ByteBuffer.wrap(it.getData())
    val frameMetadata =
        FrameMetadata(it.size.width, it.size.height, it.rotationToUser, Facing.FRONT)
    compositeDisposable.add(
        rxFaceDetectionProcessor.process(byteBuffer, frameMetadata)
            .subscribe()
    )
}

All this is done while keeping the reactive nature of the app intact via RxFaceDetectionProcessor . The ViewModel will pass on the frame to the processor which will process each frame asynchronously off the UI thread.
RxFaceDetectionProcessor is the reactive layer written over FaceDetectionProcessor that emits face-detection results via FaceDetectionResultListener which are then consumed by faceDetectionResultLiveData = LiveData<List<Face>> . This faceDetectionResultLiveData is observed via the view layer via viewModel to display the rectangular bounding box over the face.

class RxFaceDetectionProcessor
@Inject
constructor(private val faceDetectionProcessor: FaceDetectionProcessor) :
    FlowableOnSubscribe<List<Face>>,
    FaceDetectionResultListener {
    private lateinit var emitter: FlowableEmitter<List<Face>>
    private lateinit var data: ByteBuffer
    private lateinit var frameMetadata: FrameMetadata
    private lateinit var faceDetectionResultLiveData: MutableLiveData<List<Face>>

    fun setFaceDetectionResultLiveData(faceDetectionResultLiveData: MutableLiveData<List<Face>>) {
        this.faceDetectionResultLiveData = faceDetectionResultLiveData
    }

    fun process(
        data: ByteBuffer,
        frameMetadata: FrameMetadata
    ): Flowable<List<Face>> {
        this.data = data
        this.frameMetadata = frameMetadata
        return Flowable.create(this, BackpressureStrategy.LATEST)
    }

    override fun subscribe(emitter: FlowableEmitter<List<Face>>) {
        this.emitter = emitter
        faceDetectionProcessor.process(data, frameMetadata, this)
    }

    override fun onSuccess(
        results: List<Face>
    ) {
        faceDetectionResultLiveData.value = results
    }

    override fun onFailure(e: Exception) {
        Timber.d(e)
        faceDetectionResultLiveData.value = emptyList()
    }

    fun stop() {
        faceDetectionProcessor.stop()
        if (::emitter.isInitialized)
            emitter.setDisposable(Disposables.disposed())
    }
}

FaceDetectionProcessor is the class where actual frame processing happens. This involves creating FaceDetector with FaceDetectorOptions to start processing each frame for our use case.

val options = FaceDetectorOptions.Builder()
    .apply {
      setClassificationMode(FaceDetectorOptions
                                       .CLASSIFICATION_MODE_NONE)
       setLandmarkMode(FaceDetectorOptions.LANDMARK_MODE_NONE)
       setPerformanceMode(FaceDetectorOptions.PERFORMANCE_MODE_FAST)
       enableTracking()
    }
    .build()
detector = FaceDetection.getClient(options)

We wanted the face detection to be as fast as possible without requiring any extra processing on landmarks or classification. Hence, we went for faster performance by limiting the features and not requiring the classification of faces(smiling, eyes open, etc) or any landmark detection.

Why are the frames getting processed even if the face detector is closed?
When we started pushing frames for face detection, we came around with one more blocker issue. We found that even if the face detector was closed and the camera view had been destroyed, the frames kept on getting processed thus affecting the battery performance. Anything which hinders the performance, cannot hit production at any cost. So we sat around to fix the issue. We found that we were feeding more frames to faceDetector than it could actually consume and since faceDetector took time to compute the result for a particular frame, the next frame was getting processed even if the faceDetector was closed because those were already sent in its buffer. Here is the processing logic after the fix.

class FaceDetectionProcessor
@Inject
constructor() {
    private val detector: FaceDetector
    private var latestImage: ByteBuffer? = null
    private var latestImageMetaData: FrameMetadata? = null
    private var processingImage: ByteBuffer? = null
    private var processingMetaData: FrameMetadata? = null

    init {
        // Face detector initialisation here
    }

    fun process(
        data: ByteBuffer,
        frameMetadata: FrameMetadata,
        detectionResultListener: FaceDetectionResultListener
    ) {
        latestImage = data
        latestImageMetaData = frameMetadata
        // Process the image only when the last frame processing has been completed
        if (processingImage == null && processingMetaData == null) {
            processLatestImage(detectionResultListener)
        }
    }

    private fun processLatestImage(detectionResultListener: FaceDetectionResultListener) {
        processingImage = latestImage
        processingMetaData = latestImageMetaData
        latestImage = null
        latestImageMetaData = null
        if (processingImage != null && processingMetaData != null) {
            processImage(
                requireNotNull(processingImage),
                requireNotNull(processingMetaData),
                detectionResultListener
            )
        }
    }

    private fun processImage(
        data: ByteBuffer,
        frameMetadata: FrameMetadata,
        detectionResultListener: FaceDetectionResultListener
    ) {
        detectInVisionImage(
            InputImage.fromByteBuffer(
                data,
                frameMetadata.width,
                frameMetadata.height,
                frameMetadata.rotation,
                InputImage.IMAGE_FORMAT_NV21
            ),
            detectionResultListener
        )
    }

    private fun detectInVisionImage(
        image: InputImage,
        detectionResultListener: FaceDetectionResultListener
    ) {
        detector.process(image)
            .addOnSuccessListener {
                detectionResultListener.onSuccess(it)
            }
            .addOnFailureListener {
                detectionResultListener.onFailure(it)
            }.addOnCompleteListener {
                // Process the next available frame for face detection
                processLatestImage(detectionResultListener)
            }
    }

    fun stop() {
        try {
            Timber.d("Face detector closed")
            detector.close()
        } catch (e: IOException) {
            Timber.e("Exception thrown while trying to close Face Detector: $e")
        }
    }
}

Quality Selfies via Auto Capture 🤳

We launched our newly developed selfie experience to our users which received very good feedback. At InCred, we always aim to make good, better and then strive for the best. So, we completely enhanced the selfie experience by introducing auto-capture once the face is detected with an utmost quality snapshot. We provided an oval overlay on top of the camera feed and asked our users to have their faces inside it. Once the face was detected, we would auto-capture the selfie. To guide the users properly, we provided real-time feedback like (You are too near to the camera/ You look too far from the camera, etc.) on top of the overlay so that our users could do this task without requiring any manual support.

Detecting the face inside the oval and the feedback

We added an oval overlay on top of the GraphicOverlay. Note that GraphicOverlay helped earlier in adding the rectangular bounding box graphic once the face is detected. We were looking at 3 kinds of feedback from the users:

• Face inside oval: To detect whether the detected face was actually inside the oval, it was just about checking whether the face bounding box was inside the oval dimensions. Here this is the oval dimensions and other parameters for the face bounding box.

    fun sidesInsideOval(top: Float, right: Float, bottom: Float, left: Float): Boolean {
        if (top >= this.top && bottom <= this.bottom && left >= this.left && right <= this.right)
            return true
        return false
    }
  

• The face inside the oval but zoomed out: The face of the user could be inside the oval but if it is too far from the camera, the selfies captured would not be clear. To verify this case, we compared the vertical distance of the face bounding box with the oval. If it was less than half, we gave the feedback to the user to move closer to the camera.

    fun isFaceZoomedOut(top: Float, bottom: Float): Boolean {
        if (((bottom - top) / (this.bottom - this.top)) <= 0.5)
            return true
        return false
    }
  

• Face zoomed in: The user could be holding the camera too near to the face which could bring in sub-quality selfies. To verify this, we checked whether any of the coordinates of the face bounding box is greater than the oval dimension or not. If the face was zoomed in, we gave the feedback to the user to move near to the camera.

    fun isFaceZoomedIn(top: Float, right: Float, bottom: Float, left: Float): Boolean {
        var sidesInside = 0
        if (top >= this.top)
            sidesInside += 1
        if (bottom <= this.bottom)
            sidesInside += 1
        if (left >= this.left)
            sidesInside += 1
        if (right <= this.right)
            sidesInside += 1
        if (sidesInside <= 1)
            return true
        return false
    }
  

Once the corner cases had been tested, we experimented with our new selfie experience in conjunction with the old one via Firebase A/B testing. Within a month we found that the oval overlay with the auto-capture feature was performing way better than the former one. This provided confidence in our new feature and then we launched it to all our users.

The fallback approach

Android is heavily fragmented and with different OEMs, it is more and more difficult to test your feature on all Android devices. This becomes more problematic in a country like India where there is a vast majority of OEMs that customize the behaviour of Android OS according to their needs and requirements. This means that if your feature does not work on some devices, you are basically blocking the user journey in your app. To solve for those users, we provided a fallback approach where if a face was not detected in 10s, we would move to the native camera experience. With this, we were not blocking the user’s loan application journey at any cost while giving us time to fix the issues in the background.

Always provide the fallback approach for your feature. At the end, you would never want to block your users and then receive bad reviews on Play Store.

We truly believe that Necessity is the mother of invention. The current tough times are pushing us to re-imagine and build every feature without requiring any human intervention. We are on our path to not only rebuild/revamp our tech stack but also solidify and improve the features that can boost the user experience. If this excites you, we will be more than happy to discuss it.

Hunt for an API

We would like to build a REST API on top of BigQuery which can tell the app status for a given user on the given month range. At InCred, language is not the barrier where a microservice can be written in Java/GoLang/NodeJs. For this case, we went with NodeJs and used BigQuery NodeJs client along the way. The API would have the following request body:

{
  "userIds": [
  <list of user identifiers>
  ],
  "startMonth": 1,
  "startYear": 2020,
  "endMonth": 5,
  "endYear": 2020
}

For every user, this API should return the status of our app with values ranging from Uninstalled/Installed/Never Installed.

Our Android app fires many events to Firebase Analytics. These events help in analyzing user behaviour. To gain deeper insights, we have BigQuery integrated into Firebase analytics. To track uninstalls, we started looking out whether we can explicitly fire any event when the app is getting uninstalled by bundling it with user information. Unfortunately, this was not available and then we came across Firebase auto-logged events, where one of the events app_remove was specifically meant for that purpose. To define the uniqueness of a user, we send [user_id](https://firebase.google.com/docs/analytics/userid)when the users authenticate via phone. Right, so just querying with app_remove event name will not solve the problem. That wasn’t meant to be!!

What if the user had reinstalled the app again?

If we just went with listening to theapp_remove event, the user having reinstalled the app, will be shown as Uninstalled. Not the desired result. We also wanted to check if we could find anything unique about the user before logging into our app and then we saw user_pseudo_id. Firebase automatically generates this unique ID within BigQuery for each user. So, we created a map of user_id and user_pseudo_id and then analyzed the uninstalls. This failed too as it started giving false-positive results.

Keep-It-Simple-Silly(KISS)

When the problem becomes complex, it is always good to break it down into sub-problems. Then, solve the sub-problems and connect them all together. For a user in the given month range, we broke this problem into 3 parts:

• We would get the maximum event_timestamp for the app_remove event. This would also club the fact that the user has installed/uninstalled the app multiple times. Since BigQuery shards the table based on date, we used wildcard(*) to cover all the tables for a particular month.

    select max(event_timestamp) as app_remove_timestamp, user_id as user_id from " + <table_name.yyyymm*> + " where event_name = 'app_remove' and user_id is not null
  

• We would get the maximum engagement timestamp of a user. This would get the last timestamp when the user engaged with our app.

    select max(event_timestamp) as max_engage_timestamp, user_id as user_id from " + <table_name.yyyymm*> + " where user_id is not null
  

• Both the results would also be run on the same day since the BigQuery intraday table name is a little different. We would combine the last results to give us the bucket for Installed/Uninstalled/Never Installed users. Let’s see how:

  • If the app_remove_timestamp is greater than max_engage_timestamp, then the user has Uninstalled the app.

  • If the app_remove_timestamp is null or less than max_engage_timestamp, then the user has the app Installed

  • If both the timestamps are null, then the user has Never Installed the app in that range.

Note: All this works for a given month range. We know for sure that when the user has taken a loan from us and then gives an estimated month range on this API to give the desired result.

Why not include a date too to make this API complete?

With month ranges, we were not able to track the uninstalls every week or in the given time frame, say, we wanted to track from 5th Jan 2020 to 3rd Mar 2020 when we ran a campaign or disbursed loans via some offer. To cater to that, we changed the request body a little to include optional startDay and endDay and then tweaked the earlier solution. With the help of TABLE_SUFFIX, we were able to scan the table for the given date range.

For the range of 5th Jan 2020 to 3rd Mar 2020, we used TABLE_SUFFIX to get the data set from 5th Jan to 30th Jan, then ran a Wildcard(*) for Feb month, and lastly ran a range query with TABLE_SUFFIX to query from 1st Mar to 3rd Mar.

With this, we successfully developed an API over BigQuery that can track user uninstalls on the given date range.

Periodic notifications via Airflow

Once we developed an API, we wanted to utilize it the most by pushing automatic reminders to our users to have the app installed. At InCred, we use Apache Airflow to program, schedule, and monitor our workflows. Airflow provides a very neat dashboard to monitor your jobs, logging, retry policy, and much more via DAG runs.

Let’s break down the problem. We would get the users with active loans(SQL query to our database), run those users on our Uninstall tracking API, and then send notifications via SMS to all those who have uninstalled our Android app.

Since the users with active loans would be huge, running every logic on a single DAG would take a lot of time to complete. Also, if there are a huge number of users then our SQL query underneath BigQuery would significantly increase thus hitting the Maximum unresolved standard SQL query length of 1 MB. So, we broke this problem even further and created multiple child DAGs.

• We set our master DAG to get the users having active loans.

• We configured the batch size on Airflow to create a child DAG for every batch.

• These child DAGs would run in parallel to get install status and send notifications.

• Child DAGs would have their retry policy along with master DAG.

Once we brainstormed on the solution, with Airflow, creating this setup was a piece of cake. We did just that and executed our workflow to work like a charm. We then went ahead and scheduled it bi-weekly for periodic reminders to our users.

Change is the only constant — Mahatma Gandhi

Two years back when TvFlix repository was created, the aim was to follow the latest tech stack and architecture. So the app was first designed in MVP pattern and then last year migrated to MVVM by Android Architecture Components powered by RxJava and Dagger 2. Now, the complete TvMaze repository has been rewritten in Kotlin with the latest tech stack and language goodness. So let’s dive in.

Upgrade to Moshi

TvMaze earlier relied upon Gson for converting JSON to POJO and back. To promote immutability, AutoValue was being used with Parcelable and Gson extensions by Ryan Harter. Gson TypeAdapters helped in avoiding reflection while making the conversion. AutoValue also helped in creating builders while still maintaining immutability. A typical AutoValue class will look like this:

Too much of a boilerplate isn’t it?! Also, since the libraries are moving to Kotlin first then why not upgrade for a better future with the goodness of the language? Moshi solves all of the above to stand out as a better JSON serialization library. It provides first-hand Kotlin support with moshi-kotlin and codegen. Kotlin data classes are immutable and copy the method helps in creating the builder. Kotlin's @Parcelize helps to replace the AutoValue parcel extension thus removing the complete AutoValue library. So the above data class becomes pretty neat and small.

@Parcelize
@JsonClass(generateAdapter = true)
data class Episode(
val id: Long,
val url: String?) : Parcelable

Migration to Coroutine from RxJava — A Paradigm Shift

Coroutines are new ways to encourage asynchronous programming. They help in synchronously writing asynchronous operations. There are tons of articles that already explain the Whys and Hows of Coroutines. So let’s jump directly into implementation.

With Retrofit latest release, Coroutine is officially supported. That means you can remove retrofit’s RxJava2CallAdapterFactory . Just make all the retrofit calls to suspend functions and return the values directly.

interface TvMazeApi {
    @GET("/schedule")
    suspend fun getCurrentSchedule(
        @Query("country") country: String,
        @Query("date") date: String
    ): List<Episode>

    @GET("/shows")
    suspend fun getShows(@Query("page") pageNumber: Int): List<Show>
}

The home screen of TvMaze shows the list of shows fetched from the TVDB API clubbing them with the user’s favorites. A user can mark any show as a favorite which gets stored via Room. Room library now comes with Coroutine support which means all the Dao operations can be suspended thus removing RxJava completely.

Now we are ready to load data for Home screen. HomeViewModel previously relied on RxJava heavily. When the onScreenCreated was called, two streams: one from API and the other from DB were zipped using Single.zipto give the combined result.

public void onScreenCreated() {
    isLoading.setValue(true);  
    String currentDate = getCurrentDate();   

    Single<List<Show>> favoriteShows = favoriteShowsRepository
    .getAllFavoriteShows()
    .map(this::convertToShows);       

    Single<List<Episode>> episodes = tvMazeApi.getCurrentSchedule(COUNTRY_US, currentDate);
    Single<List<Episode>> zippedSingleSource = 
    Single.zip(episodes, favoriteShows, this::favorites);
Disposable homeDisposable = zippedSingleSource.observeOn(AndroidSchedulers.mainThread())
    .subscribeOn(Schedulers.io())
    .subscribe(this::onSuccess, this::onError);
    compositeDisposable.add(homeDisposable);   
}

Let's convert the above function to coroutines. Since Retrofit and Room are already on Coroutine support, they respect the suspend calls. That means, viewModelScope will launch a new coroutine, which then calls retrofit’s suspend function tvMazeApi.getCurrentSchedule. The retrofit will move the network operation to the IO-bound coroutine via Dispatchers.IO and once the suspending function is complete, it will return the result on Dispatchers.Main. A similar case will happen with Room suspend calls.

fun onScreenCreated() {
    homeViewStateLiveData.value = Loading
    val coroutineExceptionHandler = CoroutineExceptionHandler { _, exception ->
        onError(exception)
    }
    // viewModelScope launches the new coroutine on Main Dispatcher internally
    viewModelScope.launch(coroutineExceptionHandler) {
        // Get favorite shows from db, suspend function in room will launch a new coroutine with IO dispatcher
        val favoriteShowIds = favoriteShowsRepository.allFavoriteShowIds()
        // Get shows from network, suspend function in retrofit will launch a new coroutine with IO dispatcher
        val episodes = tvMazeApi.getCurrentSchedule(COUNTRY_US, currentDate)

        // Return the result on main thread via Dispatchers.Main
        homeViewStateLiveData.value = Success(HomeViewData(getShowsWithFavorites(episodes, favoriteShowIds)))
    }
}

viewModelScope is a lifecycle Kotlin extension to support coroutines. This handles the cancellation of the coroutine when ViewModel onCleared is called so that you don’t have to worry about holding the Job instance and explicitly canceling it. CoroutineExceptionHandler handles the exceptions being thrown at any Coroutine so that the errors can be propagated and displayed to the user.

Unit Testing Coroutine

We have introduced Coroutine to replace RxJava both on the network and DB layer. Time to put them to the test. We will use Mockito with mockito-kotlin for better mocking APIs. The test will run on RobolectricTestRunner to incorporate Android resources into testing.

Testing Room With Coroutine

TvFlix stores the user-marked favorite shows in Db. All the CRUD operations happen via FavoriteShowsRepository

To test FavoriteShowsRepository, we will create an in-memory database via Room.inMemoryDatabaseBuilder. The in-memory database provides faster CRUD operations on the database and also ensures that the database disappears when the process is killed. To effectively test coroutines, we will rely on Coroutine test library(still in development). Let’s check the FavoriteShowsRepositoryTest class and understand the flow:

We mock the context and provide it to Room to create the database. Mocking of context is possible by mockito-inline which helps in mocking final classes. With the database, we get the mock showDao and create our repository. runBlocking runs the new coroutine on the current thread until its completion. We launch the new Coroutine or run the suspending function in this block. We have used Truth library which provides better and more expressive assertions.

Testing ViewModel

InstantTaskExecutorRule is an Android architecture testing component that executes the task in the same thread. MainCoroutineRule sets the main Coroutines Dispatcher to TestCoroutineScope for better control over coroutine execution in unit testing. HomeViewModel is dependent upon TvMazeApi and FavoriteShowsRepository. So, we mock the required dependencies and then use @InjectMocks to mock HomeViewModel. To test, if the home screen is loaded with shows and without favorites, we first stub network and DB calls. Then, we verify whether the loader is shown or not. pauseDispatcher pauses the execution of a coroutine to verify the initial Loading state. After verification, we resumeDispatcher to proceed with execution of pending coroutine actions.

Initially, when HomeViewModelTest was run, it failed because of the way the tests were written. Read more here. Then after further readings and listening to an awesome Podcast, it was found that we don’t explicitly have to worry about context-switching with Retrofit and Room. The libraries that come with Coroutine support, own the suspend functions and move them off the UI thread on their own. Thus, they reduce the hassles of handling thread callbacks by the developers in a big way. The result is pretty neat asynchronous calls written synchronously with no callbacks and better readability.

UI Testing — Robot Pattern

Robot Pattern was introduced by Jake Wharton to provide a better separation of concern for UI testing. The pattern aims at properly managing the UI test by separating the What and How of it. Meaning that what is to be tested should not concern How it is to be tested. Both should have clear separation so that maintenance of UI tests becomes easy.

Say, we want to test the Home Screen, whether it shows the list of Popular Shows, and whether Add to Favorites is working fine or not. This is the What of Robot Pattern. The How of this will be taken care of by HomeRobot which handles all the verification/assertions/actions via Espresso.

LoadingIdlingResource tells Espresso to wait until the list ofshows is fetched and displayed on the screen. This is achieved via the visibility of the progress bar. As an alternative, some suggest using Espresso in production code, but having a test code in production does not look good(Debatable). The final piece is our Kotlin Robot HomeRobot

This Robot is loaded with Kotlin goodness to remove the builder pattern and use language advanced primitives. Know more here.

Future is Bright

RxJava is a powerful library for asynchronous programming packed with tons of operators to get along. Coroutines are still new, and the Android community is slowly accepting them. Heavy development around Flow and Channels, are bridging this gap. Kotlin surely bags a punch with superb language features. With an awesome community, the road to the future of Android Application development looks even brighter. So why wait, go ahead and give a boost to your Android development with Kotlin now. 🚀