How to delete dynamically created datagridview columns in c#?

Question

Im unable to delete columns that i had created during run time. How do I delete dynamically created columns?

Here is the code to create columns in a button :

           //creating new column
            DataGridViewTextBoxColumn column1 = new DataGridViewTextBoxColumn();
            column1.ReadOnly = false;
            column1.Name = "column"+ incrementcount;
            column1.Resizable = DataGridViewTriState.True;
            //column1.SortMode = DataGridViewColumnSortMode.Automatic;
            column1.HeaderCell.Style.Alignment = DataGridViewContentAlignment.MiddleCenter;
            column1.DefaultCellStyle.Alignment = DataGridViewContentAlignment.MiddleCenter;
            column1.HeaderText = "Column" + incrementcount;                
            column1.Width = 100;
            dataGridView4.AutoGenerateColumns = false;
            dataGridView4.Columns.Add(column1);

And im trying to delete the column like this, I also tried to hide the column :

           // dataGridView4.Columns[poss].Visible = false;
           dataGridView4.Columns.Remove(dataGridView4.Columns[positionindex].HeaderText);
           dataGridView4.Refresh();

But still the columns are visible. How do I delete it?