Using AI for object classification.

In this post I will show you the easiest way to combine AI, convolution neural network(CNN) and docker container to classified object in real time. So all thing you need to know is basic knowledge about docker and neural network. If you are very new to programming, don’t worry, just follow the step below, and you will have a program classified object in real time.

in the video above I’m driving a car go around with a camera on top, to tracking other car and person inside it. I use CUDA Yolo + Nvidia GPU. You can also do the same, all you need to do is download my Docker file and run it.

For who need to understand the theories behind, I will summaries like this. The docker file will create a Ubuntu Linux environment and install Nvidia GPU+OpenCV+darknet in to it. Darknet is a wonderful neural network, it was train by around 10 millions picture and can real-time recognize about 70 categories (car, dog, cat, ship, plane….). If you want to learn more about darknet, you can read my article : https://thanhnguyensite.net/2020/11/05/neural-network/

OK! now let’s go the AI world:

Darknet Nvidia-Docker Ubuntu 16.04

Prerequisites

  1. Make sure you have the NVidia driver for your machine

Find out your the Graphics Card model

lspci | grep VGA

https://www.nvidia.com/Download/index.aspx?lang=en-us

How to install NVidia Drivers on Linux https://gist.github.com/wangruohui/df039f0dc434d6486f5d4d098aa52d07#install-nvidia-graphics-driver-via-runfile

  1. Install Docker and NVidia Docker https://github.com/NVIDIA/nvidia-docker

Steps to run

  1. Clone this repo:
git clone https://gitlab.com/thanhnguyen1181991/darknet-docker.git
  1. Build the machine (this step might take a while, go make some coffee)
docker build -t darknet .
  1. On start.sh make sure you have the correct address of your webcam, in file start.sh line 8, if you use laptop onboard webcam, then choose: device=/dev/bus/usb/003/004:/dev/video0, if use external webcam, then: device=/dev/bus/usb/003/004:/dev/video0

Find your webcam bus

lsusb -t

Change the following line with the correct webcam bus

--device=/dev/bus/usb/003/002:/dev/video0

  1. Map a local folder to the Docker Container

Format:

/local/folder:/docker/folder

on start.sh change the following line

-v /home/projects:/dev/projects \
  1. Run the machine with Webcam
sh start.sh

Darknet

Make sure you have the weights for what you want to run

More information at https://pjreddie.com/darknet/

Metal as a Services(MaaS)

Hello again! Today’s class is metal as a service MAAS so we’ve been doing a number of classes lately on these different services, we’ve talked about software as a service we’ve talked about infrastructure as a service, and we have talked about platform as a service. Now when you’re talking about these services by far the most popular solution out there is software as a service that is where you basically you go out and you lease software from companies and the software is all run on their server so we’re talking about software as a service think about things like Google Docs think about things like Salesforce the software is not installed on your local computer.

Its installed on their servers and you access that somehow these are either through a web browser or through some type of a terminal connection something like that tied by the infrastructure as a service basically when your your infrastructure all those things that you would buy and install in your premises you no longer actually own anymore.

So think about your telephone systems companies you should spend $50 thousand on their telephone system and that telephone system was installed in their premises and they owned it now you can get hosted voice over IP solutions such as on sip you get hosted firewall solutions you can have hosted server rooms why have your own servers when you can go to Amazon Web Services and simply spin up a number of virtual services on their platform so that is infrastructure as a service.

We then talked about platform as a service platform as a service is where you create your web apps and then you are looking for some place to host them so the basic idea is think about a shared hosting web plan you create your web site you create your web application and then you simply upload that to GoDaddy or one in one servers their servers have PHP installed their servers have my sequel installed their servers have Ruby or Perl or any of these other scripting languages that you need all that you have to worry about is your application of course that gets much more complicated once you go over to Google App Engine and some of the more and more advanced things but that’s the the basic concept so now we’re getting to basically probably the last service that I will be talking about is metal as a service now this is one of those really really really cool ideas that actually it is pretty cutting-edge I’m not sure if it’s bleeding edge but it’s pretty cutting edge so some of you guys looking to create businesses out there really should listen to what I’m talking about right now because it really is a good business opportunity right now because not very many people actually offer this service so when we’re talking about metal as a service what we are literally talking about is providing the server hardware as a service to clients so this is not the same thing as what you would normally think about with cloud computing or virtualization where you install a hypervisor onto a server and then they spin up some kind of a instance of a server and this isn’t the same thing as a dedicated server so for Eli the computer guy comm and a couple of other things that I do we rent a dedicated server from a company called one and one comm with that I get a specific server 12 gigs of ram quad core processor blah blah blah with a certain version of Linux on it and then from that point I can configure up but with that when you are purchasing something like a dedicated server you have to use whatever operating systems are provided by the provider so with one and one com if you go with them you can use like Server 2008 or Server 2012 Fedora sent OS if you wanted to use something else tough luck if you wanted to use freebsd on one of these services that you couldn’t do it if you wanted to install a hypervisor on Twenties dedicated services servers you couldn’t do it the reason is is because although you’re renting the dedicated server it has to have a bare minimum installation on it before you are actually able to get access to it the cool thing with metal as a service it is the concept is that you are literally renting the physical box with nothing else on it so this is where you would go to a provider and you would literally rent it would be a quad-core xeon processor with so much RAM so much hard drive and that’s it there would not necessarily be sent OS on it there were not necessarily be windows on it there wouldn’t necessarily be anything on it you are you are literally renting the metal as a service so basically now instead of having to have your own server rooms with your own equipment your own server racks your own HVAC you’re all redundant to power supply and your ups and all of that kind of stuff you can have the same thing sitting in somebody else’s server room so they are renting to you the metal as a service now that you may be wondering why why would you bother with that if you can get virtual servers if you can use infrastructure as a service if you can even get dedicated servers why would you run want to rent or lease the bare metal as a service well as you go through with your companies if you have a startup company or if you have a technology company and you start growing what you are going to find is no matter what operating system distribution you use it is probably not going to be optimized for whatever it is you’re doing so you know we all know with Windows we all know with Windows Windows hogs up a lot of extra resources to do things most of us really truly don’t care about it we’ll be happy if it did it but a lot of people don’t realize is even with Linux even with Linux there are resources that are used there are security vulnerabilities that are opened up simply because when you install the default installation of whatever Linux you’re going to be using it installs a base level of applications and services and a lot of times you don’t need that so imagine if you were a company where you want to spin out a lot of database servers and you want those database servers to run at the absolute optimum the fastest they can possibly run well you may want to go in to a distribution of an operating system and literally rip out all the crap that you don’t want you don’t need notepad you don’t need tar you don’t need a lot of these these things you just need that server to run as fast as possible to do a specific task possibly do something like a database server because this becomes very important especially when you start dealing with larger companies that are dealing with a larger load on their servers because when you rip out all of the crap you don’t need on it on a servers operating system you can gain efficiencies now this is not you’re not going to probably speed up the server by 200% or 500% or a thousand percent right that’s not what what the target you’re going to hit you may be able to actually speed up the server though by something like five percent now for you especially if you haven’t dealt with real server rooms if you haven’t dealt with real loads on servers optimizing an operating system we get five percent improvement probably doesn’t sound like a big deal but with companies if you if you have 20 servers up and running or 40 servers up and running a five percent increase in speed can be very very very very very very very significant so with these companies they may be looking to optimize the things like I say the operating systems that will be installed how all this will be configured how all this will be managed and so all they want is the bare metal they want the server they want the hard drives they want the RAM they want the CPU but they don’t want anybody else to mess with the rest of it they want to be able to build this thing from the ground up and again there can be a lot of reasons for this nowadays things like again efficiency making sure that the resources on the server are optimized but also issues such as compliance so compliance is becoming a bigger and bigger deal within the the corporate world what compliance means is that you are running your IT systems to certain specifications for security and reliability so as more and more companies start using cloud computers and servers and all that to run the infrastructure of their business they have to make sure that that infrastructure is reliable enough for their industry now one of the problems if you go out and you use a standard instance of an operating system or you use a standard load of an operating system from one of these providers is you don’t necessarily know all the security flaws you haven’t necessarily been able to sit down and do penetration test and do hardening testing and do all of those things so when you when you are leasing let’s say from 1 + 1 , dedicated server you can’t guarantee that this is that the the server operating system that has been installed is as hardened as it should be now again for somebody like me I don’t care again do good backups and you should be fine for and this is one of the things you have to think about for 98% of the business population this type of concept doesn’t matter but for that 2% it is very very very important it is very important that they know that whatever operating system and software that’s going to be installed in that server it lives up to certain specifications so that’s why they would want to be able to rent that that bare metal as a service so metal as a service now one of the questions that you’re going to be coming up in with Eli’s you’re gonna be saying work and got me saying well Eli uh I don’t understand how you would interact with metal as a service then because you know when we think about dealing with these virtual computers when we think about dealing with it with cloud computers and all that we have a basic interface to deal with so basically again if you get a dedicated server you get a virtual private server they spin up the operating system and then they give you the login credentials so basically the company that you’re buying the service from they have already installed the instance of the operating system they’ve already created the first user account they then give you that information for that first user account and then you can figure it out however it is you want so the question you may be asking say well I don’t I don’t understand that because if you’re literally renting the metal and the metal let’s say five states away well wait a minute but there’s no operating system to interact with and the metals five states away so you don’t want to drive there so so I don’t understand how you would configure it or work with it well one of the cool things and not really new but but they’re they’re coming too more into vogue is something called IPP KVM switches so kate KVM switches been around for forever long far longer than i’ve been in the computer industry keyboard video mouse switches so what these are generally when you’re dealing with a server rack is you plug all the servers in the rack into one KVM switch and then you can press a button then when you press that button that gives you access to the server from one keyboard video and mouse combination so you have a monitor you have a keyboard and mouse and you say oh I need to deal with the wit server – and you just hit the server – button and server – pops up oh I need to deal with server 10 you click the server 10 button and the server 10 pops up well with KVM they now have IP k via what this means is that you can deal with that server from the basic input/output so the basic video keyboard and mouse and you can do that over an IP connection so you can either open that up through a web browser and be able to log in or you can open it up through some kind of terminal session or or some kind of application so basically you can be sitting in your in your office five states away from this bare metal the company that you’re dealing with will plug in the kV em switch and whatever else and then basically you can hit the on there they can hit the on button or you may have some kind of remote wait at the on button and from there it will literally load into a BIOS screen then from there depending on what the the service provider has for you you can go and you can install your your your your operating system and do all of your configurations but literally you have remote access to the lowest level of that server so you could literally reboot that server and go into the BIOS and change BIOS configuration settings you literally have that ability whether you’re five states away or you know on an entirely different continent now especially with PDUs so the the power distribution units basically what most people will call surge protectors even those have remote access so that you can do things like power cycle the server because again the question where you’re like well Eli I don’t understand if if you have metal as a service if you have that metal and you do something wrong and it freezes up how do you force it to restart because again you know you’re installing operating systems you’re doing all kind of wacky stuff sometimes it’ll freeze and if you have access to the metal what are you going to do well with these surge protectors these power distribution units you can actually power cycle them again remotely so this is the cool stuff with metal as a service I think this is going to become a much more prominent thing right now this is one of those things that it is offered by companies you don’t see it around a lot but it is something that you should be looking at and you should be thinking about because again this way you can have you you can have your own custom servers that have been hardened up to your specifications but they are sitting in somebody else’s data center you don’t have to worry about it just like with all these commoditized items it is less expensive for you to be able to rent this service from somebody else that can have a thousand or five thousand or twenty thousand of these servers up and running basically they can have five or ten technicians running around making sure all the metal is doing with metal supposed to do versus if you had servers in each one of your individual offices and having people run around and deal with that kind of stuff so that’s the basic concept of metal as a service again all it is at the end of the day is you are literally leasing or renting that bare metal so you’ve got a server with absolutely no operating system on it that is what you’re leasing that allows you to do a lot of really cool sexy amazing cool stuff um and with the modern technology like I say it’s actually very very very doable today it’s one of those things when you got to start thinking we think about the cloud I mean that’s a whole one of the problems with us old technicians right is we’re used to touching stuff we like you know when we work on computers we’re used to keyboards we’re used to like plugging away and we’ve got the server in front of us and we got the router in front of us we got all this stuff in front of us so like mentally we think about all this equipment like being in our server room being in our office being in our facility and so what you have to realize is in this modern world that we’re in you can have the exact same functionality that you would have if the equipment was in your building but it can be somewhere else it can be provisioned given to you very quickly it can be given to you very inexpensively and you can be provided as securely or more securely than what you could do yourself again a lot of people you know I’m starting to talk you know talking about things like metal as a service and everybody gets worried about security everybody’s like oh how do I how do I know Eli how do I know my servers are going to be secure how do I know that data center is going to be secure well one you do something called due diligence you you actually make sure that the company that you’re dealing with is a legitimate company you probably if you’re going to be running your business office stuff you should fly out to their data center at least once to actually take a look at it make sure they’ve got all the security stuff and all that but beyond that what a lot of people don’t realize is how in secure their facilities actually are they always worry about how insecure the cloud provider is and they somehow completely ignore just the crappy crappy crappy crappy crappy security that they have on their facility again I’m here in Baltimore Maryland in the Baltimore City we have an incredibly high crime rate and so one of the real problems that you have is you can have antivirus on your servers you can have your firewalls on your servers you can of your intrusion detection on the servers you can have your ups on the servers and some crackhead could break into your building literally rip the server out of server rack and walk away with it and try to sell for 25 bucks to the local pawn shop and when they can’t sell it at the local pawn shop then they’ll get pissed and they’ll just throw it in a ditch and keep walking again that’s the nice thing with these data centers at least with that kind of physical security you would be surprised many times they have much much much much better physical security in the rest of this then then you have take a real honest hard look at the security in your facility and if you’re honest about it you probably know that it’s it’s probably pretty bad it’s probably probably probably your servers would be better off in some kind of hosted solution so that’s all there is for from metal as a service.

I enjoy taking this class and look forward to see you the next one you.

Linux for noob(like me)

In this post I will share experience using linux command, and was we can do and play with a operating system for Developer, or shorter is Linux. You don’t need to know more about this, I will show you how you can create funny program with some Linux command, what you can’t do in Windows.

1. Linux updates and upgrades do not require you to reboot

Re-booting after every software installation or update is very annoying in Windows. I keep wondering why this is not necessary in Linux, but on Windows is rule of tumb – every installation is asking you to re-boot, or after you download the Upgrades annoying window pop up and say – will reboot in 10 minutes, save your work. This is really annoying.

With Linux all you need to do is run : sudo apt update

Then have a cup of coffee until it update your computer.

2. No need to install drivers every time you plug in your computer USB device

OK, I understand that there are custom devices which use uncommon drivers like printers, cameras, etc, but why on earth every time you plug in Windows simple mass storage disk drive, or USB-Serial converter, or Mouse, or Keyboard – devices which are standard and are embedded in every Windows OS after W98 it ALWAYS ask for drivers which are already there? Why on Linux I plug the external HHD or Flash drive and it automatically mounts on my computer and I can work with it, while Windows is asking me for drivers and several minutes scans and show me different windows with warnings and Continue buttons like I’m doing something scaring which may ruin my OS?

3. You can move image/boot-able drive between machines without need for reinstall

Yes! this is something windows users can’t imagine is possible! I do remember back in the dark ages Windows asked me to re-install after I have upgraded the RAM memory size! Now imagine you get your Windows boot-able HDD and plug it on other computer, will it boot? no way!

4. system config in files not registers mess

Now this is one of the most annoying WIndows features – after several months of installing and removing software your registers and windows/system directory becomes so bloated with shared DLLs and mess that some people start making money by writing registry cleaning software!

5. You can’t boot windows from USB Stick

Probably they didn’t find a way to ask you for registration and to collect your money every time they sense this USB is plug to other computer??

6. You cant see this on Windows:

# uptime
15:54PM up 122 days, 11:22, 5 users, load average: 0.12, 0.30, 0.13

every few days if you do not reboot windows machine it starts to act slowly due to the severe memory fragmentation

Python MySQL

Python MySQL – Create Database

Last Updated: 09-03-2020

Python Database API ( Application Program Interface ) is the Database interface for the standard Python. This standard is adhered to by most Python Database interfaces. There are various Database servers supported by Python Database such as MySQL, GadFly, mSQL, PostgreSQL, Microsoft SQL Server 2000, Informix, Interbase, Oracle, Sybase etc. To connect with MySQL database server from Python, we need to import the mysql.connector interface.

Syntax:

CREATE DATABASE DATABASE_NAME
importing rquired libraries
 import mysql.connector 
 dataBase = mysql.connector.connect( 
 host ="localhost", 
 user ="user", 
 passwd ="gfg"
 ) 
 preparing a cursor object
 cursorObject = dataBase.cursor() 
 creating database
 cursorObject.execute("CREATE DATABASE geeks4geeks") 

Output:

python-mysql-create-db

The above program illustrates the creation of MySQL database geeks4geeks in which host-name is localhost, the username is user and password is gfg.

Let’s suppose we want to create a table in the database, then we need to connect to a database. Below is a program to create a table in the geeks4geeks database which was created in the above program.

importing required library
 import mysql.connector 
 connecting to the database
 dataBase = mysql.connector.connect( 
                     host = "localhost", 
                     user = "user", 
                     passwd = "gfg", 
                     database = "geeks4geeks" ) 
 preparing a cursor object
 cursorObject = dataBase.cursor() 
 creating table
 studentRecord = """CREATE TABLE STUDENT ( 
                 NAME VARCHAR(20) NOT NULL, 
                 BRANCH VARCHAR(50), 
                 ROLL INT NOT NULL, 
                 SECTION VARCHAR(5), 
                 AGE INT, 
                 )"""
 table created
 cursorObject.execute(studentRecord) 
 disconnecting from server
 dataBase.close() 

Output:

PYTHON-MYSQL-CREATE-DB1

Python MongoDB

MongoDB Python | Insert and Update Data

Prerequisites : MongoDB Python Basics
We would first understand how to insert a document/entry in a collection of a database. Then we would work on how to update an existing document in MongoDB using pymongo library in python. The update commands helps us to update the query data inserted already in MongoDB database collection.

Insert data

We would first insert data in MongoDB.

Step 1 – Establishing Connection: Port number Default: 27017conn = MongoClient(‘localhost’, port-number)If using default port-number i.e. 27017. Alternate connection method:conn = MongoClient()

Step 2 – Create Database or Switch to Existing Database:db = conn.dabasenameCreate a collection or Switch to existing collection:

collection = db.collection_name

Step 3 – Insert : To Insert Data create a dictionary object and insert data in database. Method used to insert data: insert_one() or insert_many()After insert to find the documents inside a collection we use find() command. The find() method issues a query to retrieve data from a collection in MongoDB. All queries in MongoDB have the scope of a single collection.
Note : ObjectId is different for every entry in database collection.
Let us understand insert of data with help on code:

# Python code to illustrate 
# inserting data in MongoDB 
from pymongo import MongoClient 
  
try: 
    conn = MongoClient() 
    print("Connected successfully!!!") 
except:   
    print("Could not connect to MongoDB") 
  
# database 
db = conn.database 
  
# Created or Switched to collection names: my_gfg_collection 
collection = db.my_gfg_collection 
  
emp_rec1 = { 
        "name":"Mr.Geek", 
        "eid":24, 
        "location":"delhi"
        } 
emp_rec2 = { 
        "name":"Mr.Shaurya", 
        "eid":14, 
        "location":"delhi"
        } 
  
# Insert Data 
rec_id1 = collection.insert_one(emp_rec1) 
rec_id2 = collection.insert_one(emp_rec2) 
  
print("Data inserted with record ids",rec_id1," ",rec_id2) 
  
# Printing the data inserted 
cursor = collection.find() 
for record in cursor: 
    print(record) 
Connected successfully!!!
Data inserted with record ids    
{'_id': ObjectId('5a02227b37b8552becf5ed2a'), 
'name': 'Mr.Geek', 'eid': 24, 'location': 'delhi'}
{'_id': ObjectId('5a02227c37b8552becf5ed2b'), 'name':
'Mr.Shaurya', 'eid': 14, 'location': 'delhi'}

Updating data in MongoDB

Methods used: update_one() and update_many()
Parameters passed:
+ a filter document to match the documents to update
+ an update document to specify the modification to perform
+ an optional upsert parameter

# Python code to illustrate 
# updating data in MongoDB 
# with Data of employee with id:24 
from pymongo import MongoClient 
  
try: 
    conn = MongoClient() 
    print("Connected successfully!!!") 
except:   
    print("Could not connect to MongoDB") 
  
# database 
db = conn.database 
  
# Created or Switched to collection names: my_gfg_collection 
collection = db.my_gfg_collection 
  
# update all the employee data whose eid is 24 
result = collection.update_many( 
        {"eid":24}, 
        { 
                "$set":{ 
                        "name":"Mr.Geeksforgeeks"
                        }, 
                "$currentDate":{"lastModified":True} 
                  
                } 
        ) 
  
  
  
print("Data updated with id",result) 
  
# Print the new record 
cursor = collection.find() 
for record in cursor: 
    print(record) 

Output:

Connected successfully!!!
Data updated with id 
{'_id': ObjectId('5a02227b37b8552becf5ed2a'), 
'name': 'Mr.Geeksforgeeks', 'eid': 24, 'location': 
'delhi', 'lastModified': datetime.datetime(2017, 11, 7, 21, 19, 9, 698000)}
{'_id': ObjectId('5a02227c37b8552becf5ed2b'), 'name': 
'Mr.Shaurya', 'eid': 14, 'location': 'delhi'}

To find number of documents or entries in collection the are updated use.

 print(result.matched_count) 

Python OpenCV

OpenCV is one of the most popular computer vision libraries. If you want to start your journey in the field of computer vision, then a thorough understanding of the concepts of OpenCV is of paramount importance.
In this article, I will try to introduce the most basic and important concepts of OpenCV in an intuitive manner.
This article will cover the following topics:

  1. Reading an image
  2. Extracting the RGB values of a pixel
  3. Extracting the Region of Interest (ROI)
  4. Resizing the Image
  5. Rotating the Image
  6. Drawing a Rectangle
  7. Displaying text

This is the original image that we will manipulate throughout the course of this article.

Original Image

Let’s start with the simple task of reading an image using OpenCV.

Reading an image

Importing the OpenCV library
 import cv2 
 Reading the image using imread() function
 image = cv2.imread('image.png') 
 Extracting the height and width of an image
 h, w = image.shape[:2] 
 Displaying the height and width
 print("Height = {}, Width = {}".format(h, w)) 

Now we will focus on extracting the RGB values of an individual pixel.
Note – OpenCV arranges the channels in BGR order. So the 0th value will correspond to Blue pixel and not Red.

Extracting the RGB values of a pixel

Extracting RGB values.
 Here we have randomly chosen a pixel
 by passing in 100, 100 for height and width.
 (B, G, R) = image[100, 100] 
 Displaying the pixel values
 print("R = {}, G = {}, B = {}".format(R, G, B)) 
 We can also pass the channel to extract
 the value for a specific channel
 B = image[100, 100, 0] 
 print("B = {}".format(B)) 
Region Of Interest

Python Examples

Basic Programs:

  1. Python program to add two numbers
  2. Maximum of two numbers in Python
  3. Python Program for factorial of a number
  4. Python Program for simple interest
  5. Python Program for compound interest
  6. Python Program to check Armstrong Number
  7. Python Program for Program to find area of a circle
  8. Python program to print all Prime numbers in an Interval
  9. Python program to check whether a number is Prime or not
  10. Python Program for n-th Fibonacci number
  11. Python Program for How to check if a given number is Fibonacci number?
  12. Python Program for n\’th multiple of a number in Fibonacci Series
  13. Program to print ASCII Value of a character
  14. Python Program for Sum of squares of first n natural numbers
  15. Python Program for cube sum of first n natural numbers

Array Programs:

  1. Python Program to find sum of array
  2. Python Program to find largest element in an array
  3. Python Program for array rotation
  4. Python Program for Reversal algorithm for array rotation
  5. Python Program to Split the array and add the first part to the end
  6. Python Program for Find reminder of array multiplication divided by n
  7. Python Program to check if given array is Monotonic

List Programs:

  1. Python program to interchange first and last elements in a list
  2. Python program to swap two elements in a list
  3. Python | Ways to find length of list
  4. Python | Ways to check if element exists in list
  5. Different ways to clear a list in Python
  6. Python | Reversing a List
  7. Python program to find sum of elements in list
  8. Python | Multiply all numbers in the list
  9. Python program to find smallest number in a list
  10. Python program to find largest number in a list
  11. Python program to find second largest number in a list
  12. Python program to find N largest elements from a list
  13. Python program to print even numbers in a list
  14. Python program to print odd numbers in a List
  15. Python program to print all even numbers in a range
  16. Python program to print all odd numbers in a range
  17. Python program to print positive numbers in a list
  18. Python program to print negative numbers in a list
  19. Python program to print all positive numbers in a range
  20. Python program to print all negative numbers in a range
  21. Remove multiple elements from a list in Python
  22. Python – Remove empty List from List
  23. Python | Cloning or Copying a list
  24. Python | Count occurrences of an element in a list
  25. Python | Remove empty tuples from a list
  26. Python | Program to print duplicates from a list of integers
  27. Python program to find Cumulative sum of a list
  28. Python | Sum of number digits in List
  29. Break a list into chunks of size N in Python
  30. Python | Sort the values of first list using second list

Python OOP

A class is a user-defined blueprint or prototype from which objects are created. Classes provide a means of bundling data and functionality together. Creating a new class creates a new type of object, allowing new instances of that type to be made. Each class instance can have attributes attached to it for maintaining its state. Class instances can also have methods (defined by its class) for modifying its state.
To understand the need for creating a class let’s consider an example, let’s say you wanted to track the number of dogs which may have different attributes like breed, age. If a list is used, the first element could be the dog’s breed while the second element could represent its age. Let’s suppose there are 100 different dogs, then how would you know which element is supposed to be which? What if you wanted to add other properties to these dogs? This lacks organization and it’s the exact need for classes.
Class creates a user-defined data structure, which holds its own data members and member functions, which can be accessed and used by creating an instance of that class. A class is like a blueprint for an object.
Some points on Python class:
Classes are created by keyword class.
Attributes are the variables that belong to class.
Attributes are always public and can be accessed using dot (.) operator. Eg.: Myclass.Myattribute


<strong>Class Definition Syntax:</strong> class ClassName:     
# Statement-1     
.     .     .     
# Statement-N 
<strong>Defining a class –</strong>

python class

Declaring Objects (Also called instantiating a class)
When an object of a class is created, the class is said to be instantiated. All the instances share the attributes and the behavior of the class. But the values of those attributes, i.e. the state are unique for each object. A single class may have any number of instances.
Example:

python declaring an object

Declaring an object


# Python program to
# demonstrate instantiating
# a class


classDog: 

# A simple class
# attribute
attr1 ="mamal"
attr2 ="dog"

# A sample method  
deffun(self): 
print("I'm a", self.attr1)
print("I'm a", self.attr2)

# Driver code
# Object instantiation
Rodger =Dog()

# Accessing class attributes
# and method through objects
print(Rodger.attr1)
Rodger.fun()

Output:
mamal I’m a mamal I’m a dog
In the above example, an object is created which is basically a dog named Rodger. This class only has two class attributes that tell us that Rodger is a dog and a mammal.
The self
Class methods must have an extra first parameter in method definition. We do not give a value for this parameter when we call the method, Python provides it.
If we have a method which takes no arguments, then we still have to have one argument.
This is similar to this pointer in C++ and this reference in Java.
When we call a method of this object as myobject.method(arg1, arg2), this is automatically converted by Python into MyClass.method(myobject, arg1, arg2) – this is all the special self is about.
__init__ method:
The __init__ method is similar to constructors in C++ and Java. Constructors are used to initialize the object’s state. Like methods, a constructor also contains a collection of statements(i.e. instructions) that are executed at the time of Object creation. It is run as soon as an object of a class is instantiated. The method is useful to do any initialization you want to do with your object.

# A Sample class with init method 
classPerson: 
# init method or constructor  
def__init__(self, name): 
self.name =name 
# Sample Method  
defsay_hi(self): 
print('Hello, my name is', self.name)
p =Person('Nikhil') 
p.say_hi() 

 
Output:
Hello, my name is Nikhil
Class and Instance Variables
Instance variables are for data unique to each instance and class variables are for attributes and methods shared by all instances of the class. Instance variables are variables whose value is assigned inside a constructor or method with self whereas class variables are variables whose value is assigned in the class.
Defining instance varibale using constructor.

# Python program to show that the variables with a value  
# assigned in the class declaration, are class variables and 
# variables inside methods and constructors are instance 
# variables. 

# Class for Dog 
classDog: 

# Class Variable 
animal ='dog'

# The init method or constructor 
def__init__(self, breed, color): 

# Instance Variable     
self.breed =breed
self.color =color        

# Objects of Dog class 
Rodger =Dog("Pug", "brown") 
Buzo =Dog("Bulldog", "black") 

print('Rodger details:')   
print('Rodger is a', Rodger.animal) 
print('Breed: ', Rodger.breed)
print('Color: ', Rodger.color)

print('\nBuzo details:')   
print('Buzo is a', Buzo.animal) 
print('Breed: ', Buzo.breed)
print('Color: ', Buzo.color)

# Class variables can be accessed using class 
# name also 
print("\nAccessing class variable using class name")
print(Dog.animal)        

Output:
Rodger details: Rodger is a dog Breed: Pug Color: brown Buzo details: Buzo is a dog Breed: Bulldog Color: black Accessing class variable using class name dog
Defining instance variable using the normal method.

# Python program to show that we can create  
# instance variables inside methods 

# Class for Dog 
classDog: 

# Class Variable 
animal ='dog'

# The init method or constructor 
def__init__(self, breed): 

# Instance Variable 
self.breed =breed             

# Adds an instance variable  
defsetColor(self, color): 
self.color =color 

# Retrieves instance variable     
defgetColor(self):     
returnself.color    

# Driver Code 
Rodger =Dog("pug") 
Rodger.setColor("brown") 
print(Rodger.getColor())  
Output:
brown

Python CSV

Working with csv files in Python

csv3

This article explains how to load and parse a CSV file in Python.

First of all, what is a CSV ?
CSV (Comma Separated Values) is a simple file format used to store tabular data, such as a spreadsheet or database. A CSV file stores tabular data (numbers and text) in plain text. Each line of the file is a data record. Each record consists of one or more fields, separated by commas. The use of the comma as a field separator is the source of the name for this file format.

For working CSV files in python, there is an inbuilt module called csv.

Reading a CSV file

# importing csv module 
import csv 

# csv file name 
filename = "aapl.csv"

# initializing the titles and rows list 
fields = [] 
rows = [] 

# reading csv file 
with open(filename, 'r') as csvfile: 
	# creating a csv reader object 
	csvreader = csv.reader(csvfile) 
	
	# extracting field names through first row 
	fields = next(csvreader) 

	# extracting each data row one by one 
	for row in csvreader: 
		rows.append(row) 

	# get total number of rows 
	print("Total no. of rows: %d"%(csvreader.line_num)) 

# printing the field names 
print('Field names are:' + ', '.join(field for field in fields)) 

# printing first 5 rows 
print('\nFirst 5 rows are:\n') 
for row in rows[:5]: 
	# parsing each column of a row 
	for col in row: 
		print("%10s"%col), 
	print('\n') 

The output of above program looks like this:

csv1

The above example uses a CSV file aapl.csv which can be downloaded from here.
Run this program with the aapl.csv file in same directory.

Let us try to understand this piece of code.

  • with open(filename, ‘r’) as csvfile: csvreader = csv.reader(csvfile)Here, we first open the CSV file in READ mode. The file object is named as csvfile. The file object is converted to csv.reader object. We save the csv.reader object as csvreader.
  • fields = csvreader.next()csvreader is an iterable object. Hence, .next() method returns the current row and advances the iterator to the next row. Since the first row of our csv file contains the headers (or field names), we save them in a list called fields.
  • for row in csvreader: rows.append(row)Now, we iterate through remaining rows using a for loop. Each row is appended to a list called rows. If you try to print each row, one can find that row is nothing but a list containing all the field values.
  • print(“Total no. of rows: %d”%(csvreader.line_num))csvreader.line_num is nothing but a counter which returns the number of rows which have been iterated.

Writing to a CSV file

# importing the csv module 
import csv 

# field names 
fields = ['Name', 'Branch', 'Year', 'CGPA'] 

# data rows of csv file 
rows = [ ['Nikhil', 'COE', '2', '9.0'], 
		['Sanchit', 'COE', '2', '9.1'], 
		['Aditya', 'IT', '2', '9.3'], 
		['Sagar', 'SE', '1', '9.5'], 
		['Prateek', 'MCE', '3', '7.8'], 
		['Sahil', 'EP', '2', '9.1']] 

# name of csv file 
filename = "university_records.csv"

# writing to csv file 
with open(filename, 'w') as csvfile: 
	# creating a csv writer object 
	csvwriter = csv.writer(csvfile) 
	
	# writing the fields 
	csvwriter.writerow(fields) 
	
	# writing the data rows 
	csvwriter.writerows(rows)

Let us try to understand the above code in pieces.

  • fields and rows have been already defined. fields is a list containing all the field names. rows is a list of lists. Each row is a list containing the field values of that row.
  • with open(filename, ‘w’) as csvfile: csvwriter = csv.writer(csvfile)Here, we first open the CSV file in WRITE mode. The file object is named as csvfile. The file object is converted to csv.writer object. We save the csv.writer object as csvwriter.
  • csvwriter.writerow(fields)Now we use writerow method to write the first row which is nothing but the field names.
  • csvwriter.writerows(rows)We use writerows method to write multiple rows at once.

Writing a dictionary to a CSV file

# importing the csv module 
import csv 

# my data rows as dictionary objects 
mydict =[{'branch': 'COE', 'cgpa': '9.0', 'name': 'Nikhil', 'year': '2'}, 
		{'branch': 'COE', 'cgpa': '9.1', 'name': 'Sanchit', 'year': '2'}, 
		{'branch': 'IT', 'cgpa': '9.3', 'name': 'Aditya', 'year': '2'}, 
		{'branch': 'SE', 'cgpa': '9.5', 'name': 'Sagar', 'year': '1'}, 
		{'branch': 'MCE', 'cgpa': '7.8', 'name': 'Prateek', 'year': '3'}, 
		{'branch': 'EP', 'cgpa': '9.1', 'name': 'Sahil', 'year': '2'}] 

# field names 
fields = ['name', 'branch', 'year', 'cgpa'] 

# name of csv file 
filename = "university_records.csv"

# writing to csv file 
with open(filename, 'w') as csvfile: 
	# creating a csv dict writer object 
	writer = csv.DictWriter(csvfile, fieldnames = fields) 
	
	# writing headers (field names) 
	writer.writeheader() 
	
	# writing data rows 
	writer.writerows(mydict) 

In this example, we write a dictionary mydict to a CSV file.

  • with open(filename, ‘w’) as csvfile: writer = csv.DictWriter(csvfile, fieldnames = fields)Here, the file object (csvfile) is converted to a DictWriter object.
    Here, we specify the fieldnames as an argument.
  • writer.writeheader()writeheader method simply writes the first row of your csv file using the pre-specified fieldnames.
  • writer.writerows(mydict)writerows method simply writes all the rows but in each row, it writes only the values(not keys).

So, in the end, our CSV file looks like this:

csv2

Important Points:

  • In csv modules, an optional dialect parameter can be given which is used to define a set of parameters specific to a particular CSV format. By default, csv module uses excel dialect which makes them compatible with excel spreadsheets. You can define your own dialect using register_dialect method.
    Here is an example:
     csv.register_dialect(
    'mydialect',
    delimiter = ',',
    quotechar = '"',
    doublequote = True,
    skipinitialspace = True,
    lineterminator = '\r\n',
    quoting = csv.QUOTE_MINIMAL)

Now, while defining a csv.reader or csv.writer object, we can specify the dialect like
this:

csvreader = csv.reader(csvfile, dialect='mydialect')
  • Now, consider that a CSV file looks like this in plain-text:
    We notice that the delimiter is not a comma but a semi-colon. Also, the rows are separated by two newlines instead of one. In such cases, we can specify the delimiter and line terminator as follows:csvreader = csv.reader(csvfile, delimiter = ';', lineterminator = '\n\n')

Data Types

Python | Set 3 (Strings, Lists, Tuples, Iterations)

Last Updated: 20-10-2020

In the previous article, we read about the basics of Python. Now, we continue with some more python concepts.

Strings in Python 
A string is a sequence of characters. It can be declared in python by using double-quotes. Strings are immutable, i.e., they cannot be changed.

Assigning string to a variable
 a = "This is a string"
 print (a) 

Lists in Python 
Lists are one of the most powerful tools in python. They are just like the arrays declared in other languages. But the most powerful thing is that list need not be always homogeneous. A single list can contain strings, integers, as well as objects. Lists can also be used for implementing stacks and queues. Lists are mutable, i.e., they can be altered once declared.

# Declaring a list 
L = [1, "a" , "string" , 1+2] 
print L 
L.append(6) 
print L 
L.pop() 
print L 
print L[1] 

The output is :  

[1, 'a', 'string', 3]
[1, 'a', 'string', 3, 6]
[1, 'a', 'string', 3]
a


Tuples in Python 
A tuple is a sequence of immutable Python objects. Tuples are just like lists with the exception that tuples cannot be changed once declared. Tuples are usually faster than lists.

Example

Create a Tuple:

thistuple = ("apple", "banana", "cherry")
print(thistuple)