diff --git a/2022/es/Days/day30.md b/2022/es/Days/day30.md index ba1abab..c33eeb1 100644 --- a/2022/es/Days/day30.md +++ b/2022/es/Days/day30.md @@ -1,165 +1,165 @@ -## Microsoft Azure Security Models +## Modelos de seguridad de Microsoft Azure -Following on from the Microsoft Azure Overview, we are going to start with Azure Security and see where this can help in our day to day. For the most part, I have found the built-in roles have been sufficient but knowing that we can create and work with many different areas of authentication and configurations. I have found Microsoft Azure to be quite advanced with its Active Directory background compared to other public clouds. +Siguiendo con la visión general de Microsoft Azure, vamos a empezar con Azure Security y ver cómo esto puede ayudar. En gran medida, con los roles por defecto son suficientes, pero además podemos trabajar con muchas áreas diferentes de autenticación y configuraciones. Microsoft Azure puede ser bastante avanzado gracias a Active Directory en comparación con otras nubes públicas. -This is one area in which Microsoft Azure seemingly works differently from other public cloud providers, in Azure there is ALWAYS Azure AD. +Esta es un área en la que Microsoft Azure aparentemente funciona de manera diferente a otros proveedores de nube pública, en Azure SIEMPRE tiene Active Directory. -### Directory Services +### Servicios de directorio -- Azure Active Directory hosts the security principles used by Microsoft Azure and other Microsoft cloud services. -- Authentication is accomplished through protocols such as SAML, WS-Federation, OpenID Connect and OAuth2. -- Queries are accomplished through REST API called Microsoft Graph API. -- Tenants have a tenant.onmicrosoft.com default name but can also have custom domain names. -- Subscriptions are associated with an Azure Active Directory tenant. +- Azure Active Directory alberga los principios de seguridad utilizados por Microsoft Azure y otros servicios en la nube de Microsoft. +- La autenticación se realiza a través de protocolos como SAML, WS-Federation, OpenID Connect y OAuth2. +- Las consultas se realizan a través de la API REST denominada Microsoft Graph API. +- Los tenants tienen un nombre por defecto tenant.onmicrosoft.com pero también pueden tener nombres de dominio personalizados. +- Las suscripciones están asociadas a un tenant de Azure Active Directory. -If we think about AWS to compare the equivalent offering would be AWS IAM (Identity & Access Management) Although still very different +Si pensamos en AWS para comparar el servicio equivalente sería AWS IAM (Identity & Access Management), aunque es bastante diferente -Azure AD Connect provides the ability to replicate accounts from AD to Azure AD. This can also include groups and sometimes objects. This can be granular and filtered. Supports multiple forests and domains. +Azure AD Connect ofrece la posibilidad de replicar cuentas de AD a Azure AD. Esto también puede incluir grupos y a veces objetos. Esto puede ser granulado y filtrado. Admite varios bosques y dominios. -It is possible to create cloud accounts in Microsoft Azure Active Directory (AD) but most organisations already have accounted for their users in their own Active Directory being on-premises. +Es posible crear cuentas en la nube en Microsoft Azure Active Directory (AD), pero la mayoría de las organizaciones ya tienen contabilizados a sus usuarios en su propio Active Directory local. -Azure AD Connect also allows you to not only see Windows AD servers but also other Azure AD, Google and others. This also provides the ability to collaborate with external people and organisations this is called Azure B2B. +Azure AD Connect también permite ver no sólo los servidores Windows AD sino también otros Azure AD, Google y otros. Esto también proporciona la capacidad de colaborar con personas y organizaciones externas, lo que se denomina Azure B2B. -Authentication options between Active Directory Domain Services and Microsoft Azure Active Directory are possible with both identity sync with a password hash. +Las opciones de autenticación entre Active Directory Domain Services y Microsoft Azure Active Directory son posibles con ambas identidades sincronizadas con un hash de contraseña. ![](Images/Day30_Cloud1.png) -The passing of the password hash is optional, if this is not used then pass-through authentication is required. +El paso del hash de la contraseña es opcional, si esto no se utiliza entonces se requiere autenticación passthrough. -There is a video linked below that goes into detail about Passthrough authentication. +Hay un vídeo enlazado a continuación que entra en detalle sobre la autenticación Passthrough. -[User sign-in with Azure Active Directory Pass-through Authentication](https://docs.microsoft.com/en-us/azure/active-directory/hybrid/how-to-connect-pta) +[Inicio de sesión de usuario con autenticación Pass-through de Azure Active Directory](https://docs.microsoft.com/en-us/azure/active-directory/hybrid/how-to-connect-pta) ![](Images/Day30_Cloud2.png) -### Federation +### Federación -It's fair to say that if you are using Microsoft 365, Microsoft Dynamics and on-premises Active Directory it is quite easy to understand and integrate into Azure AD for federation. However, you might be using other services outside of the Microsoft ecosystem. +Es justo decir que utilizando Microsoft 365, Microsoft Dynamics y Active Directory local es bastante fácil de entender e integrar en Azure AD para la federación. Sin embargo, es posible utilizar otros servicios fuera del ecosistema de Microsoft. -Azure AD can act as a federation broker to these other Non-Microsoft apps and other directory services. +Azure AD puede actuar como intermediario de federación para estas otras aplicaciones ajenas a Microsoft y otros servicios de directorio. -This will be seen in the Azure Portal as Enterprise Applications of which there are a large number of options. +Esto se verá en el Portal Azure como Aplicaciones Empresariales de las cuales hay un gran número de opciones. ![](Images/Day30_Cloud3.png) -If you scroll down on the enterprise application page you are going to see a long list of featured applications. +Desplazando hacia abajo la página de aplicaciones empresariales se puede obtener una larga lista de aplicaciones destacadas. ![](Images/Day30_Cloud4.png) -This option also allows for "bring your own" integration, an application you are developing or a non-gallery application. +Esta opción también permite "traer su propia" integración, una aplicación que está desarrollando o una aplicación no galería. -I have not looked into this before but I can see that this is quite the feature set when compared to the other cloud providers and capabilities. +No he mirado en esto antes, pero puedo ver que esto es bastante el conjunto de características en comparación con los otros proveedores de nube y capacidades. -### Role-Based Access Control +### Control de acceso basado en roles -We have already covered on [Day 29](day29.md) the scopes we are going to cover here, we can set our role-based access control according to one of these areas. +Ya hemos cubierto en [Día 29](day29.md) los ámbitos que vamos a cubrir aquí, podemos establecer nuestro control de acceso basado en roles de acuerdo a uno de estos ámbitos: -- Subscriptions -- Management Group -- Resource Group -- Resources +- Suscripciones +- Grupo de Gestión +- Grupo de Recursos +- Recursos -Roles can be split into three, there are many built-in roles in Microsoft Azure. Those three are: +Los roles se pueden dividir en tres, hay muchos roles incorporados en Microsoft Azure. Estos tres son: -- Owner -- Contributor -- Reader +- Propietario +- Contribuidor +- Lector -Owner and Contributor are very similar in their boundaries of scope however the owner can change permissions. +Propietario y Contribuidor son muy similares en sus límites de alcance. Sin embargo, el propietario puede cambiar permisos. -Other roles are specific to certain types of Azure Resources as well as custom roles. +Otros roles son específicos para ciertos tipos de Azure Resources, así como roles personalizados. -We should focus on assigning permissions to groups vs users. +Deberíamos centrarnos en asignar permisos a grupos frente a usuarios. -Permissions are inherited. +Los permisos se heredan. -If we go back and look at the "90DaysOfDevOps" Resource group we created and check the Access Control (IAM) within you can see we have a list of contributors and a customer User Access Administrator, and we do have a list of owners (But I cannot show this) +Si volvemos atrás y miramos el grupo de Recursos "90DaysOfDevOps" creado y comprobamos el Control de Acceso (IAM) dentro podemos ver que tenemos una lista de contribuidores y un cliente Administrador de Acceso de Usuario, y tenemos una lista de propietarios (Pero no puedo mostrar esto) ![](Images/Day30_Cloud5.png) -We can also check the roles we have assigned here if they are BuiltInRoles and which category they fall under. +Podemos comprobar aquí si los roles que tenemos asignados son BuiltInRoles y a qué categoría pertenecen. ![](Images/Day30_Cloud6.png) -We can also use the check access tab if we want to check an account against this resource group and make sure that the account we wish to have that access to has the correct permissions or maybe we want to check if a user has too much access. +También podemos comprobar acceso si queremos comprobar una cuenta contra este grupo de recursos y asegurarnos de que la cuenta a la que queremos dar ese acceso tiene los permisos correctos o quizás queremos comprobar si un usuario tiene demasiado acceso. ![](Images/Day30_Cloud7.png) ### Microsoft Defender for Cloud -- Microsoft Defender for Cloud (formerly known as Azure Security Center) provides insight into the security of the entire Azure environment. +- Microsoft Defender for Cloud (anteriormente conocido como Azure Security Center) proporciona información sobre la seguridad de todo el entorno Azure. -- A single dashboard for visibility into the overall security health of all Azure and non-Azure resources (via Azure Arc) and security hardening guidance. +- Un único panel de control para la visibilidad del estado general de la seguridad de todos los recursos Azure y no Azure (a través de Azure Arc) y orientación sobre el refuerzo de la seguridad. -- Free tier includes continuous assessment and security recommendations. +- El nivel gratuito incluye evaluación continua y recomendaciones de seguridad. -- Paid plans for protected resource types (e.g. Servers, AppService, SQL, Storage, Containers, KeyVault). +- Planes de pago para tipos de recursos protegidos (por ejemplo, Servidores, AppService, SQL, Almacenamiento, Contenedores, KeyVault). -I have switched to another subscription to view the Azure Security Center and you can see here based on very few resources that I have some recommendations in one place. +He cambiado a otra suscripción para ver el Centro de Seguridad de Azure y se puede ver aquí sobre la base de muy pocos recursos con algunas recomendaciones en un solo lugar. ![](Images/Day30_Cloud8.png) ### Azure Policy -- Azure Policy is an Azure native service that helps to enforce organizational standards and assess compliance at scale. +- Azure Policy es un servicio nativo de Azure que ayuda a aplicar las normas de la organización y evaluar el cumplimiento a escala. -- Integrated into Microsoft Defender for Cloud. Azure Policy audits non-compliant resources and applies remediation. +- Integrado en Microsoft Defender for Cloud. Azure Policy audita los recursos no conformes y aplica correcciones. -- Commonly used for governing resource consistency, regulatory compliance, security, cost, and management standards. +- Se utiliza habitualmente para regular la coherencia de los recursos, el cumplimiento normativo, la seguridad, los costes y las normas de gestión. -- Uses JSON format to store evaluation logic and determine whether a resource is compliant or not, and any actions to take for non-compliance (e.g. Audit, AuditIfNotExists, Deny, Modify, DeployIfNotExists). +- Utiliza el formato JSON para almacenar la lógica de evaluación y determinar si un recurso es conforme o no, así como las medidas que deben tomarse en caso de incumplimiento (por ejemplo, Audit, AuditIfNotExists, Deny, Modify, DeployIfNotExists). -- Free for use. The exception is Azure Arc connected resources charged per server/month for Azure Policy Guest Configuration usage. +- Uso gratuito. La excepción son los recursos conectados a Azure Arc que se cobran por servidor/mes para el uso de Azure Policy Guest Configuration. -### Hands-On +### Manos a la obra -I have gone out and I have purchased www.90DaysOfDevOps.com and I would like to add this domain to my Azure Active Directory portal, [Add your custom domain name using the Azure Active Directory Portal](https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/add-custom-domain) +He comprado www.90DaysOfDevOps.com y quisiera añadir el dominio al portal Azure Active Directory, [Añada su nombre de dominio personalizado utilizando el portal Azure Active Directory](https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/add-custom-domain) ![](Images/Day30_Cloud9.png) -With that now, we can create a new user on our new Active Directory Domain. +Con eso ahora, podemos crear un nuevo usuario en nuestro nuevo dominio de Active Directory. ![](Images/Day30_Cloud10.png) -Now we want to create a group for all of our new 90DaysOfDevOps users in one group. We can create a group as per the below, notice that I am using "Dynamic User" which means Azure AD will query user accounts and add them dynamically vs assigned which is where you manually add the user to your group. +Ahora queremos crear un grupo para todos los nuevos usuarios 90DaysOfDevOps en un grupo. Podemos crear un grupo como el siguiente. Tenga en cuenta que estoy usando "Usuario dinámico", lo que significa que Azure AD consultará las cuentas de usuario y agregarlos dinámicamente.Asignado es donde se agrega manualmente el usuario al grupo. ![](Images/Day30_Cloud11.png) -There are lots of options when it comes to creating your query, I plan to simply find the principal name and make sure that the name contains @90DaysOfDevOps.com. +Hay muchas opciones a la hora de crear la consulta, como buscar el nombre de la entidad de seguridad y asegurar de que el nombre contiene @90DaysOfDevOps.com. ![](Images/Day30_Cloud12.png) -Now because we have created our user account already for michael.cade@90DaysOfDevOps.com we can validate the rules are working. For comparison I have also added another account I have associated to another domain here and you can see that because of this rule our user will not land in this group. +Ahora que ya hemos creado la cuenta de usuario para michael.cade@90DaysOfDevOps.com podemos validar que las reglas funcionan. Para comparar se ha añadido otra cuenta asociada a otro dominio y podéis ver que debido a esta regla nuestro usuario no aterrizará en este grupo. ![](Images/Day30_Cloud13.png) -I have since added a new user1@90DaysOfDevOps.com and if we go and check the group we can see our members. +Se añade un nuevo user1@90DaysOfDevOps.com y comprobando el grupo podemos ver a todos los miembros. ![](Images/Day30_Cloud14.png) -If we have this requirement x100 then we are not going to want to do this all in the console we are going to want to take advantage of either bulk options to create, invite, and delete users or you are going to want to look into PowerShell to achieve this automated approach to scale. +Si tenemos este requisito x100 no querremos hacer todo esto en la consola, podemos tomar ventaja de cualquiera de las opciones a granel para crear, invitar y eliminar usuarios o lo haremos en PowerShell para lograr este enfoque automatizado a escala. -Now we can go to our Resource Group and specify that on the 90DaysOfDevOps resource group we want the owner to be the group we just created. +Ahora podemos ir a nuestro grupo de recursos y especificar que en el grupo de recursos 90DaysOfDevOps queremos que el propietario sea el grupo que acabamos de crear. ![](Images/Day30_Cloud15.png) -We can equally go in here and deny assignments access to our resource group as well. +Igualmente podemos denegar también el acceso de las asignaciones a nuestro grupo de recursos. -Now if we log in to the Azure Portal with our new user account, you can see that we only have access to our 90DaysOfDevOps resource group and not the others seen in previous pictures because we do not have the access. +Si entramos en el Azure Portal con la nueva cuenta de usuario, podemos ver que sólo tenemos acceso a nuestro grupo de recursos 90DaysOfDevOps y no a los otros vistos en imágenes anteriores. ![](Images/Day30_Cloud16.png) -The above is great if this is a user that has access to resources inside of your Azure portal, not every user needs to be aware of the portal, but to check access we can use the [Apps Portal](https://myapps.microsoft.com/) This is a single sign-on portal for us to test. +Lo anterior está muy bien si se trata de un usuario que tiene acceso a los recursos dentro de su portal de Azure, no todos los usuarios necesitan conocer el portal, pero para comprobar el acceso podemos utilizar el [Apps Portal](https://myapps.microsoft.com/) Es un portal de inicio de sesión único para que podamos probar. ![](Images/Day30_Cloud17.png) -You can customise this portal with your branding and this might be something we come back to later on. +Puedes personalizar este portal con tu marca y esto podría ser algo a lo que volveremos más adelante. -## Resources +## Recursos - [Hybrid Cloud and MultiCloud](https://www.youtube.com/watch?v=qkj5W98Xdvw) - [Microsoft Azure Fundamentals](https://www.youtube.com/watch?v=NKEFWyqJ5XA&list=WL&index=130&t=12s) - [Google Cloud Digital Leader Certification Course](https://www.youtube.com/watch?v=UGRDM86MBIQ&list=WL&index=131&t=10s) - [AWS Basics for Beginners - Full Course](https://www.youtube.com/watch?v=ulprqHHWlng&t=5352s) -See you on [Day 31](day31.md) +Nos vemos en el [Día 31](day31.md). \ No newline at end of file diff --git a/2023.md b/2023.md index b3f7b5f..2963820 100644 --- a/2023.md +++ b/2023.md @@ -76,13 +76,13 @@ Or contact us via Twitter, my handle is [@MichaelCade1](https://twitter.com/Mich ### Runtime Defence & Monitoring -- [] ☁️ 28 > [](2023/day28.md) -- [] ☁️ 29 > [](2023/day29.md) -- [] ☁️ 30 > [](2023/day30.md) -- [] ☁️ 31 > [](2023/day31.md) -- [] ☁️ 32 > [](2023/day32.md) -- [] ☁️ 33 > [](2023/day33.md) -- [] ☁️ 34 > [](2023/day34.md) +- [✔️] ☁️ 28 > [System monitoring and auditing](2023/day28.md) +- [] ☁️ 29 > [Application level monitoring](2023/day29.md) +- [] ☁️ 30 > [Intrusion detection and anti-malware software](2023/day30.md) +- [] ☁️ 31 > [Firewalls and network protection](2023/day31.md) +- [] ☁️ 32 > [Vulnerability and patch management](2023/day32.md) +- [] ☁️ 33 > [Application whitelisting and software trust management](2023/day33.md) +- [] ☁️ 34 > [Runtime access control](2023/day34.md) ### Secrets Management diff --git a/2023/day28.md b/2023/day28.md index e69de29..b01ead0 100644 --- a/2023/day28.md +++ b/2023/day28.md @@ -0,0 +1,147 @@ +# Introduction to Runtime Defence & Monitoring + +Welcome to all the DevOps and DevSecOps enthusiasts! 🙌 + +We are here to learn about "Runtime defence". This is a huge subject, but we are not deterred by it and will learn about it together in the next 7 days. + +![](images/day28-0.png) + +This subject was split into major parts: +* Monitoring (1st and 2nd day) +* Intrusion detection +* Network defense +* Access control +* Application defense subjects (6th and 7th days) + +The goal is to get you up to a level in these subjects, where you can start to work on your own. + +Let's start 😎 + +# System monitoring and auditing + +## Why this is the first subject of the topic "Runtime defense and monitoring" subject? + +Monitoring computer systems is a fundamental tool for security teams, providing visibility into what is happening within the system. Without monitoring, security teams would be unable to detect and respond to security incidents. + +To illustrate this point, consider physical security. If you want to protect a building, you must have security personnel 24/7 at every entrance to control who is entering the building. In this same example, you are also tasked with controlling the security of everyone in the building therefore you must also have personnel all around. Of course, this is not scaling well therefore installing CCTV cameras at key places is a much better solution today. + +While scaling such physical security measures is difficult, for computer systems, it is easier to achieve through the installation of monitoring tools. Monitoring provides a basic level of control over the system, allowing security teams to detect problems, understand attack patterns, and maintain overall security. Beyond monitoring, there are additional security measures such as detection systems, which we can discuss further. + +Elaborating on this, here are the key reasons why monitoring is important for runtime security include: + +* Identifying security incidents: Monitoring can help organizations detect potential security incidents such as malware infections, unauthorized access attempts, and data breaches. + +* Mitigating risks: By monitoring for signs of security threats, organizations can take action to mitigate those risks before they lead to a breach or other security incident. + +* Complying with regulations: Many industries are subject to regulatory requirements that mandate certain security controls, including monitoring and incident response. + +* Improving incident response: Monitoring provides the necessary data to quickly identify and respond to security incidents, reducing the impact of a breach and allowing organizations to recover more quickly. + +* Gaining visibility: Monitoring provides insight into system activity, which can be used to optimize performance, troubleshoot issues, and identify opportunities for improvement. + + +## What to monitor and record? + +In theory, the ideal solution would be to log everything that is happening in the system and keep the data forever. + +However, this is not practical. Let's take a look at what needs to be monitored and what events need to be recorded. + +When monitoring cloud-based computer services, there are several key components that should be closely monitored to ensure the system is secure and operating correctly. These components include: + +Control plane logging: all the orchestration of the infrastructure is going through this control plane, it is crucial to always know who did what at the infrastructure level. It does not just enable the identification of malicious activity but also enables troubleshooting of the system. + +Operating level logs: log operating system level events to track system activity and detect any errors or security-related events, such as failed login attempts or system changes. Deeper logs contain information about which use does what on the machine level which is important for identifying malicious behavior. + +Network activity: Monitor network traffic to identify any unusual or unauthorized activity that could indicate an attack or compromise of the network. + +Application activity and performance: Monitor application activity to detect misbehavior in case the attack is coming from the application level. Performance monitoring is important to ensure that services are running smoothly and to respond to any performance issues that may arise. + +Resource utilization: Monitor the use of system resources such as CPU, memory, and disk space to identify bottlenecks or other performance issues. Unusual activity can be a result of denial of service-like attacks or attackers using computation resources for their good. + +Security configurations: Monitor security configurations, such as firewall rules and user access controls, to ensure that they are correctly configured and enforced. + +Backup and disaster recovery systems: Monitor backup and disaster recovery systems to ensure that they are operating correctly and data can be recovered in the event of a failure or disaster. + +## A practical implementation +In this part, we move from theory to practice. + +There isn't a silver bullet here, every system has its tools. We will work on Kubernetes as infrastructure with [Microservices demo](https://github.com/GoogleCloudPlatform/microservices-demo) application. + +### Control plane monitoring + +Kubernetes has an event auditing infrastructure called [audit logs](https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/). + +Kubernetes API server has a configuration called `Audit Policy` which tells the API server what to log. The log can either be stored in a file or sent to a webhook. + +We are using Minikube in our example, and for the sake of testing this, we will send the audit logs to the `stdout` of the API server (which is its log). + +```bash +mkdir -p ~/.minikube/files/etc/ssl/certs +cat < ~/.minikube/files/etc/ssl/certs/audit-policy.yaml +# Log all requests at the Metadata level. +apiVersion: audit.k8s.io/v1 +kind: Policy +rules: +- level: RequestResponse +EOF +minikube start --extra-config=apiserver.audit-policy-file=/etc/ssl/certs/audit-policy.yaml --extra-config=apiserver.audit-log-path=- +``` + +You can follow the logs with this Kubectl command: +```bash +kubectl logs kube-apiserver-minikube -n kube-system | grep audit.k8s.io/v1 +``` + +Every API operation is logged to the stream. + +Here is an example of an event "getting all secrets in default namespace": +```json +{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"RequestResponse","auditID":"8e526e77-1fd9-43c3-9714-367fde233c99","stage":"RequestReceived","requestURI":"/api/v1/namespaces/default/secrets?limit=500","verb":"list","user":{"username":"minikube-user","groups":["system:masters","system:authenticated"]},"sourceIPs":["192.168.49.1"],"userAgent":"kubectl/v1.25.4 (linux/amd64) kubernetes/872a965","objectRef":{"resource":"secrets","namespace":"default","apiVersion":"v1"},"requestReceivedTimestamp":"2023-02-11T20:34:11.015389Z","stageTimestamp":"2023-02-11T20:34:11.015389Z"} +``` + +As you can see, all key aspects of the infrastructure request are logged here (who, what, when). + +Storing this in a file is not practical. Audit logs are usually shipped to a logging system and database for later use. Managed Kubernetes services use their own "cloud logging" service to capture Kubernetes Audit logs. In native Kubernetes, you could use Promtail to ship logs to Prometheus as described [here](https://www.bionconsulting.com/blog/monitoring-and-gathering-metrics-from-kubernetes-auditlogs). + +### Resource monitoring + +Kubernetes ecosystem enables multiple ways to monitor resources and logging, however, the most common example is Prometheus (logging and event database) and Grafana (UI and dashboards). These two open-source tools are an easy one-stop shop for multiple tasks around monitoring. + +Out of the box, we will get resource monitoring Kubernetes nodes. + +Here is how we are installing it on the Minikube we started in the previous part. Make sure, you have `helm` installed before. + +```bash +helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +helm repo add grafana https://grafana.github.io/helm-charts +helm install prometheus prometheus-community/prometheus +helm install grafana grafana/grafana +kubectl expose service grafana --type=NodePort --target-port=3000 --name=grafana-np +``` + +Now, these services should be installed. + +To access Grafana UI, first, get the first password + +```bash +kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo +``` + +Then login to the UI +```bash +minikube service grafana-np --url +``` + +![](images/day28-1.png) + +After you have logged in, go to "Data sources/Prometheus" and add our Prometheus service as a source. The URL has to be set to `http://prometheus-server` and click "save & test". + +Now, to set up resource dashboards, go to the "Dashboards" side menu and choose "Import". Here you can import premade dashboard. For example node metrics can be imported by putting the number `6126` in the field `Import via grafana.com` and clicking the `Load` button. + +![](images/day28-2.png) + +Browse Grafana for more dashboards [here](https://grafana.com/grafana/dashboards/). + +# Next... + +Tomorrow we will continue to the application level. Application logs and behavior monitoring will be in focue. We will continue to use the same setup and go deeper into the rabbit hole 😄 diff --git a/2023/day29.md b/2023/day29.md index e69de29..8e86971 100644 --- a/2023/day29.md +++ b/2023/day29.md @@ -0,0 +1,131 @@ +# Recap + +Last day we discussed why monitoring, logging and auditing are the basics of runtime defense. In short: you cannot protect a live system without knowing what is happening. We built a Minikube cluster yesterday with Prometheus and Grafana. We are continuing to build over this stack today. +Let's start 😎 + +# Application logging + +Application logs are important from many perspective. This is the way operators know what is happening inside applications they run on their infrastrucutre. For the same reason, keeping application logs is important from a security perspective because they provide a detailed record of the system's activity, which can be used to detect and investigate security incidents. + +By analyzing application logs, security teams can identify unusual or suspicious activity, such as failed login attempts, access attempts to sensitive data, or other potentially malicious actions. Logs can also help track down the source of security breaches, including when and how an attacker gained access to the system, and what actions they took once inside. + +In addition, application logs can help with compliance requirements, such as those related to data protection and privacy. By keeping detailed logs, organizations can demonstrate that they are taking the necessary steps to protect sensitive data and comply with regulations. + +Loki is a component in the Grafana stack which collects logs using Promtail for Pods running in the Kubernetes cluster and stores them just as Prometheus does for metrics. + +To install Loki with Promtail on your cluster, install the following Helm chart. + +```bash +helm install loki --namespace=monitoring grafana/loki-stack +``` + +This will put a Promtail and a Loki instance in your Minikube and will start collecting logs. Note that this installation in not production grade and it is here to demonstrate the capabilities. + +You should be seeing the Pods are ready: +```bash +$ kubectl get pods | grep loki +loki-0 1/1 Running 0 8m25s +loki-promtail-mpwgq 1/1 Running 0 8m25s +``` + +Now go to your Grafana UI (just as we did yesterday): + +```bash +kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo +minikube service grafana-np --url +``` + +Take the secret of the admin password (if you haven't changed it already) and print the URL of the service, then go to the URL and log in. + +In order to see the logs in Grafana, we need to hook up Loki as a "data source" just as we did yesterday with Prometheus. + +![](images/day29-1.gif) + +Now add here a new Loki data source. + +The only thing that needs to be changed in the default configuration is the endpoint of the Loki service, in our case it is http://loki:3100, see it below: + +![](images/day29-2.png) + +Now click "Save & test" and your Grafana should be now connected to Loki. + +You can explore your logs in the "Explore" screen (click Explore in the left menu). + +To try our centralized logging system, we are going to check when Etcd container did compactization in the last hour. + +Choose Loki source on the top of the screen (left of the explore title) and switch from query builder mode (visual builder) to code. + +Add the following line in the query field: +``` +{container="etcd"} |= `compaction` +``` +and click "run query" on the top right part of the screen. + +You should see logs in your browser, like this: + +![](images/day29-3.png) + + +Voila! You have a logging system ;-) + + +# Application behavior monitoring + +We start to come over from general monitoring needs to low-level application monitoring for security purposes. A modern way to do this is to monitor fine-grade application behavior using eBPF. + +Monitoring applications with eBPF (extended Berkeley Packet Filter) is important from a security perspective because it provides a powerful and flexible way to monitor and analyze the behavior of applications and the underlying system. Here are some reasons why eBPF is important for application monitoring and security: + +1. Fine-grained monitoring: eBPF allows for fine-grained monitoring of system and application activity, including network traffic, system calls, and other events. This allows you to identify and analyze security threats and potential vulnerabilities in real-time. + +2. Relatively low overhead: eBPF has very low overhead, making it ideal for use in production environments. It can be used to monitor and analyze system and application behavior without impacting performance or reliability at scale. + +3. Customizable analysis: eBPF allows you to create custom analysis and monitoring tools that are tailored to the specific needs of your application and environment. This allows you to identify and analyze security threats and potential vulnerabilities in a way that is tailored to your unique needs. + +4. Real-time analysis: eBPF provides real-time analysis and monitoring, allowing you to detect and respond to security threats and potential vulnerabilities as they occur. This helps you to minimize the impact of security incidents and prevent data loss or other negative outcomes. + +Falco is a well-respected project which installs agents on your Kubernetes nodes and monitors applications at the eBPF level. + +In this part, we will install Falco in our Minikube and channel the data it collects to Prometheus (and eventually, Grafana). This part is based on this great [tutorial](https://falco.org/blog/falco-kind-prometheus-grafana/). + +In order to install Falco, you need to create private keys and certificates for client-server communication between the Falco and its exporter. + +We will use `falcoctl` for this, however you could generate your certificates and keys with `openssl` if you want. + +To install `falcoctl`, run the following command (if you are running Linux on amd64 CPU, otherwise check out [here](https://github.com/falcosecurity/falcoctl#installation)): +```bash +LATEST=$(curl -sI https://github.com/falcosecurity/falcoctl/releases/latest | awk '/location: /{gsub("\r","",$2);split($2,v,"/");print substr(v[8],2)}') +curl --fail -LS "https://github.com/falcosecurity/falcoctl/releases/download/v${LATEST}/falcoctl_${LATEST}_linux_amd64.tar.gz" | tar -xz +sudo install -o root -g root -m 0755 falcoctl /usr/local/bin/falcoctl +``` + +Now generate key pair: +```bash +FALCOCTL_NAME=falco-grpc.default.svc.cluster.local FALCOCTL_PATH=$PWD falcoctl tls install +``` + +We need to add Falco Helm repo and install the Falco services and the exporter: +```bash +helm repo add falcosecurity https://falcosecurity.github.io/charts +helm repo update +helm install falco falcosecurity/falco --set driver.kind=ebpf --set-file certs.server.key=$PWD/server.key,certs.server.crt=$PWD/server.crt,certs.ca.crt=$PWD/ca.crt --set falco.grpc.enabled=true,falco.grpcOutput.enabled=true +helm install falco-exporter --set-file certs.ca.crt=$PWD/ca.crt,certs.client.key=$PWD/client.key,certs.client.crt=$PWD/client.crt falcosecurity/falco-exporter +``` + +Make sure that all Falco Pods are running OK +```bash +$ kubectl get pods | grep falco +falco-exporter-mlc5h 1/1 Running 3 (32m ago) 38m +falco-mlvc4 2/2 Running 0 31m +``` + +Since Prometheus detects the exporter automatically and we already added the Prometheus data source, we can go directly to Grafana and install the [Falco dashboard](https://grafana.com/grafana/dashboards/11914-falco-dashboard/). + +Go to "Dashboard" left side menu and click import. In "Import via grfana.com" insert the ID `11914` and click "load". + +Now you should see Falco events in your Grafana! 😎 + +# Next... + +Next day we will look into how to detect attacks in runtime. See you tomorrow 😃 + + diff --git a/2023/day42.md b/2023/day42.md index e69de29..8e501c0 100644 --- a/2023/day42.md +++ b/2023/day42.md @@ -0,0 +1,73 @@ +# Day 42 - Programming Language:Introduction to Python + +Guido van Rossum created Python, a high-level, interpreted and dynamic programming language, in the late 1980s. It is widely used in range of applications, including web development, devops and data analysis, as well as artificial intelligence and machine learning. + +## Installation and Setting up the Environment: + +Python is available for download and installation on a variety of platforms, including Windows, Mac, and Linux. Python can be downloaded from [the official website](https://www.python.org/.). +![Python Website](/2023/images/day42-01.png) + +Following the installation of Python, you can configure your environment with an Integrated Development Environment (IDE) such as [PyCharm](https://www.jetbrains.com/pycharm/), [Visual Studio Code](https://code.visualstudio.com/), or IDLE (the default IDE that comes with Python). +I personally use Visual Studio Code. + +You can also use cloud environment, where you will not have to configure and install python locally, like [Replit](https://replit.com/). +![Replit Website](/2023/images/day42-02.png) + +## Basic Data Types: + +Python includes a number of built-in data types for storing and manipulating data. The following are the most common ones: + +- Numbers: integers, floating-point numbers, and complex numbers +- Strings are character sequences. +- Lists are ordered groups of elements. +- Tuples are ordered immutable collections of elements. +- Dictionaries are collections of key-value pairs that are not ordered. + +## Operations and Expressions: + +With the above data types, you can perform a variety of operations in Python, including arithmetic, comparison, and logical operations. +Expressions can also be used to manipulate data, such as combining multiple values into a new value. + +## Variables: + +A variable is declared and assigned a value in Python by using the assignment operator =. The variable is on the left side of the operator, and the value being assigned is on the right, which can be an expression like `2 + 2` or even other variables. As an example: + +``` python +a = 7 # assign variable a the value 7 +b = x + 3 # assign variable b the value of a plus 3 +c = b # assign variable c the value of b +``` + +These examples assign numbers to variables, but numbers are only one of the data types supported by Python. There is no type declaration for the variables. This is due to the fact that Python is a dynamically typed language, which means that the variable type is determined by the data assigned to it. The x, y, and z variables in the preceding examples are integer types, which can store both positive and negative whole numbers. + +Variable names are case sensitive and can contain any letter, number, or underscore ( ). They cannot, however, begin with a number. +Also, with numbers, strings are among the most commonly used data types. A string is a sequence of one or more characters. Strings are typically declared with single quotation marks, but they can also be declared with double quotation marks: + +``` python +a = 'My name is Rishab' +b = "This is also a string" +``` + +You can add strings to other strings — an operation known as "concatenation" — with the same + operator that adds two numbers: + +``` python +x = 'My name is' + ' ' + 'Rishab' +print(x) # outputs: My name is Rishab +``` + +## Printing to the console: + +The print function in Python is one of more than 60 built-in functions. It outputs text to the screen. +Let's see an example of the most famous "Hello World!": + +``` python +print('Hello World!') +``` + +The print argument is a string, which is one of Python's basic data types for storing and managing text. Print outputs a newline character at the end of the line by default, so subsequent calls to print will begin on the next line. + +## Resources: + +[Learn Python - Full course by freeCodeCamp](https://youtu.be/rfscVS0vtbw) +[Python tutorial for beginners by Nana](https://youtu.be/t8pPdKYpowI) +[Python Crash Course book](https://amzn.to/40NfY45) \ No newline at end of file diff --git a/2023/day43.md b/2023/day43.md index e69de29..c769f98 100644 --- a/2023/day43.md +++ b/2023/day43.md @@ -0,0 +1,114 @@ +# Day 43 - Programming Language: Python + +Welcome to the second day of Python, and today we will cover some more concepts: +- Loops +- Functions +- Modules and libraries +- File I/O + +## Loops (for/while): + +Loops are used to repeatedly run a block of code. + +### for loop + +Using the `for` loop, a piece of code is executed once for each element of a sequence (such as a list, string, or tuple). Here is an example of a for loop that prints each programming language in a list: + +``` python +languages = ['Python', 'Go', 'JavaScript'] + +# for loop +for language in languages: + print(language) +``` + +Output +``` +Python +Go +JavaScript +``` + +### while loop + +The `while loop` is used to execute a block of code repeatedly as long as a condition is True. Here's an example of a while loop that prints the numbers from 1 to 5: + +``` python +i = 1 +n = 5 + +# while loop from i = 1 to 5 +while i <= n: + print(i) + i = i + 1 +``` + +Output: +``` +1 +2 +3 +4 +5 +``` + +## Functions +Functions are reusable chunks of code with argument/parameters and return values. +Using the `def` keyword in Python, you can define a function. In your programme, functions can be used to encapsulate complex logic and can be called several times. +Functions can also be used to simplify code and make it easier to read. Here is an illustration of a function that adds two numbers: + +``` python +# function has two arguments num1 and num2 +def add_numbers(num1, num2): + sum = num1 + num2 + print('The sum is: ',sum) +``` + +``` python +# calling the function with arguments to add 5 and 2 +add_numbers(5, 2) + +# Output: The sum is: 9 +``` + +## Understanding Modules and Importing Libraries: +A module is a file in Python that contains definitions and statements. Modules let you arrange your code and reuse it across many apps. +The Standard Library, a sizable collection of Python modules, offers a wide range of capabilities, such as file I/O, regular expressions, and more. +Additional libraries can be installed using package managers like pip. +You must import a module or library using the import statement in order to use it in your programme. Here is an illustration of how to load the math module and calculate a number's square root using the sqrt() function: + +``` python +import math + +print(math.sqrt(16)) # 4.0 +``` + +## File I/O +File I/O is used to read and write data to and from files on disk. +The built-in Python function open() can be used to open a file, after which you can read from and write to it using methods like read() and write(). +To save system resources, you should always close the file after you are done with it. +An example of reading from a file and printing its content: + +``` python +f = open("90DaysOfDevOps.txt", "r") +print(f.read()) +f.close() +``` + +## Exception Handling: + +Exceptions are runtime errors that happen when your programme runs into unexpected circumstances, such dividing by zero or attempting to access a list element that doesn't exist. +Using a try/except block, you can manage exceptions in Python. The try block's code is run, and if an exception arises, the except block's code is run to handle the exception. + +``` python +try: + f = open("90DaysOfDevOps.txt") + try: + f.write("Python is great") + except: + print("Something went wrong when writing to the file") +``` + +## Conclusion + +That is it for today, I will see you tomorrow in Day 3 of Python! diff --git a/2023/images/day28-0.png b/2023/images/day28-0.png new file mode 100644 index 0000000..8cfcbfa Binary files /dev/null and b/2023/images/day28-0.png differ diff --git a/2023/images/day28-1.png b/2023/images/day28-1.png new file mode 100644 index 0000000..179256d Binary files /dev/null and b/2023/images/day28-1.png differ diff --git a/2023/images/day28-2.png b/2023/images/day28-2.png new file mode 100644 index 0000000..ed9b95c Binary files /dev/null and b/2023/images/day28-2.png differ diff --git a/2023/images/day29-1.gif b/2023/images/day29-1.gif new file mode 100644 index 0000000..64c9ac9 Binary files /dev/null and b/2023/images/day29-1.gif differ diff --git a/2023/images/day29-2.png b/2023/images/day29-2.png new file mode 100644 index 0000000..511c7dd Binary files /dev/null and b/2023/images/day29-2.png differ diff --git a/2023/images/day29-3.png b/2023/images/day29-3.png new file mode 100644 index 0000000..57f939b Binary files /dev/null and b/2023/images/day29-3.png differ diff --git a/2023/images/day42-01.png b/2023/images/day42-01.png new file mode 100644 index 0000000..76bb58a Binary files /dev/null and b/2023/images/day42-01.png differ diff --git a/2023/images/day42-02.png b/2023/images/day42-02.png new file mode 100644 index 0000000..ad8f29a Binary files /dev/null and b/2023/images/day42-02.png differ