AOA Forums AOA Forums AOA Forums Folding For Team 45 AOA Files Home Front Page Become an AOA Subscriber! UserCP Calendar Memberlist FAQ Search Forum Home


Go Back   AOA Forums > Hardware > Mobile Devices and Networking


Reply
 
LinkBack Thread Tools Rate Thread
  #1 (permalink)  
Old 21st February, 2005, 10:06 PM
Gizmo's Avatar
Chief BBS Administrator
BassTeroids Champion, Global Player Champion, Aim & Fire Champion, Puzzle Maniax Champion, Othello Champion, Canyon Glider Champion, Unicycle Challenge Champion, YetiSports 9: Final Spit Champion, Zed Champion
 
Join Date: May 2003
Location: Webb City, Mo
Posts: 16,178
Send a message via ICQ to Gizmo Send a message via AIM to Gizmo Send a message via MSN to Gizmo Send a message via Yahoo to Gizmo Send a message via Skype™ to Gizmo

Gigabit NICs

Got a question for all you networking gurus out there.

I have a customer who has a system using dual Gigabit NICs. The configuration is somewhat complex, but it goes like this:

There are three servers. Each server has two NICs. One NIC connects to a dedicated network shared just between the three servers. This is used for internal data replication between the servers, so that if one goes down, the others have the data necessary to take the load. The other NIC is used to connect the server to the outside world.

The NICs in question are Intel EtherPro/1000 using the 82465 chipset.

The customer is complaining that the system is intermittently slow to respond. In this particular case, I've got a custom web server application I wrote which implements a very small portion of the W3C standard. I use it because the mainstream web server apps are just too fat. When I look in my applications logs, everything appears normal. However, because my application is working at the TCP layer, I don't see anything from the network until the TCP connection is completed. In other words, if there is something going on with the NIC driver or the network itself that is resulting in delays or retries, it would be invisible to my app. The symptoms are that when this problem occurs, the client app makes a request and several seconds (sometimes several 10s of seconds) later it gets a response. During this period of time the server does not show unusual disk, cpu, or network traffic.

I am about to deplay a laptop machine with a gigabit adapter and a copy of Ethereal to sniff the network so that I can try to see what is going on, but I have the potential to end up digging through a LOT of data without finding anything.

Have any of you seen anything like this before?
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 21st February, 2005, 10:48 PM
Chief Systems Administrator
 
Join Date: September 2001
Location: Europe
Posts: 13,075

I've seen something similar before, but I'm not sure it'd apply to the setup you're having problems with.
When dealing with some Sun blades and Cisco switches, there was a negotiation problem where the Ciscos would negotate full duplex, and the Suns would negotiate half duplex. The end result was a somewhat intermittent drop in response time. It didn't help that we were load balancing across about four web servers, as it made it difficult to tell where the problem was.
__________________
Any views, thoughts and opinions are entirely my own. They don't necessarily represent those of my employer (BlackBerry).
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 21st February, 2005, 11:56 PM
XeroHouR's Avatar
Member/Contributer
 
Join Date: March 2004
Location: Osceola/Muncie IN
Posts: 1,264

this isn't due to massive collisions and jamming is it??
__________________
Sony Vaio SZ220
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 22nd February, 2005, 01:24 AM
Gizmo's Avatar
Chief BBS Administrator
BassTeroids Champion, Global Player Champion, Aim & Fire Champion, Puzzle Maniax Champion, Othello Champion, Canyon Glider Champion, Unicycle Challenge Champion, YetiSports 9: Final Spit Champion, Zed Champion
 
Join Date: May 2003
Location: Webb City, Mo
Posts: 16,178
Send a message via ICQ to Gizmo Send a message via AIM to Gizmo Send a message via MSN to Gizmo Send a message via Yahoo to Gizmo Send a message via Skype™ to Gizmo

Quote:
Originally Posted by XeroHouR
this isn't due to massive collisions and jamming is it??
Not as far as I can tell. I have been told that the switches (they are using managed Cisco switches) have not reported anything abnormal.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 22nd February, 2005, 01:24 AM
Gizmo's Avatar
Chief BBS Administrator
BassTeroids Champion, Global Player Champion, Aim & Fire Champion, Puzzle Maniax Champion, Othello Champion, Canyon Glider Champion, Unicycle Challenge Champion, YetiSports 9: Final Spit Champion, Zed Champion
 
Join Date: May 2003
Location: Webb City, Mo
Posts: 16,178
Send a message via ICQ to Gizmo Send a message via AIM to Gizmo Send a message via MSN to Gizmo Send a message via Yahoo to Gizmo Send a message via Skype™ to Gizmo

Quote:
Originally Posted by Áedán
I've seen something similar before, but I'm not sure it'd apply to the setup you're having problems with.
When dealing with some Sun blades and Cisco switches, there was a negotiation problem where the Ciscos would negotate full duplex, and the Suns would negotiate half duplex. The end result was a somewhat intermittent drop in response time. It didn't help that we were load balancing across about four web servers, as it made it difficult to tell where the problem was.
My first thought was something like this, but I've been told the switches are configured for full duplex, and according to the drivers, so are the NICs.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6 (permalink)  
Old 22nd February, 2005, 01:37 AM
XeroHouR's Avatar
Member/Contributer
 
Join Date: March 2004
Location: Osceola/Muncie IN
Posts: 1,264

collision isn't abnormal on a normal ethernet, that is unless you're running token ring. So yea, its prolly not that. Can you determine where as far as the layer the problem is? If one of your nics is messed up then it could be sending out an unnecissarily large amount of frames and overloading the network traffic
__________________
Sony Vaio SZ220
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7 (permalink)  
Old 22nd February, 2005, 01:54 AM
Gizmo's Avatar
Chief BBS Administrator
BassTeroids Champion, Global Player Champion, Aim & Fire Champion, Puzzle Maniax Champion, Othello Champion, Canyon Glider Champion, Unicycle Challenge Champion, YetiSports 9: Final Spit Champion, Zed Champion
 
Join Date: May 2003
Location: Webb City, Mo
Posts: 16,178
Send a message via ICQ to Gizmo Send a message via AIM to Gizmo Send a message via MSN to Gizmo Send a message via Yahoo to Gizmo Send a message via Skype™ to Gizmo

That's one of the things I'm hoping to answer with the sniffer. I need to try to establish the relationship between when the traffic arrives on the wire and when my app processes it.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #8 (permalink)  
Old 22nd February, 2005, 02:52 AM
Toro's Avatar
Member / Master of Toiletries
 
Join Date: May 2004
Location: On the lamb!!
Posts: 3,340
Send a message via MSN to Toro Send a message via Skype™ to Toro

and its nothing mad like it going on to power saving mde or anything completely insane like that?


yes i know nothing, used to, dont really know where it went, nevermind
__________________

"Check the Forum Rules Announcement, listed there."

AOA Team fah


Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #9 (permalink)  
Old 22nd February, 2005, 11:02 AM
Chief Systems Administrator
 
Join Date: September 2001
Location: Europe
Posts: 13,075

Quote:
Originally Posted by XeroHouR
collision isn't abnormal on a normal ethernet, that is unless you're running token ring.
Remember we're talking a fully switched network running in full duplex mode here. It would be unusual to be seeing any collisions on such a network. In fact, if you did see collisions, it would indicate that there was a problem somewhere.
__________________
Any views, thoughts and opinions are entirely my own. They don't necessarily represent those of my employer (BlackBerry).
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #10 (permalink)  
Old 22nd February, 2005, 07:31 PM
XeroHouR's Avatar
Member/Contributer
 
Join Date: March 2004
Location: Osceola/Muncie IN
Posts: 1,264

I was pretty sure that on a broadcast network, duplex or not, switched or not, there is always some level of collision. I am in my first semester of Cisco however, so I'm no expert *bow*
__________________
Sony Vaio SZ220
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #11 (permalink)  
Old 22nd February, 2005, 07:44 PM
Gizmo's Avatar
Chief BBS Administrator
BassTeroids Champion, Global Player Champion, Aim & Fire Champion, Puzzle Maniax Champion, Othello Champion, Canyon Glider Champion, Unicycle Challenge Champion, YetiSports 9: Final Spit Champion, Zed Champion
 
Join Date: May 2003
Location: Webb City, Mo
Posts: 16,178
Send a message via ICQ to Gizmo Send a message via AIM to Gizmo Send a message via MSN to Gizmo Send a message via Yahoo to Gizmo Send a message via Skype™ to Gizmo

In a fully switched network, there are only ever two devices on a segment; the computer and the switch. In other words, a fully switched network is basically a logical point-to-point configuration, although it will usually be physically arranged in some variation of a star topology. Under such conditions, there should never be any opportunity for collisions on a given segment, and the switching fabric itself will take care of ensuring that collisions do not occur upstream, where packets from multiple sources will share the same wire. Because of this, collisions should be nearly non-existent and the presence of collision alarms indicates a problem.

The nature of a switched network also presents some difficulties for packet sniffers like what I am trying to deploy. Why? Because the whole premise of a switched network is that the only traffic that should ever appear on a given segment is the traffic intended for a machine on that segment. If there is only one machine on any given segment, then how do you sniff traffic for an arbitrary machine and be certain that you are getting ALL of it?

There are two possibilites:

1) Insert a hub between the machine you want to sniff and the switch. Since a hub replicates ALL of its traffic to ALL of its nodes, this means that you can sniff the traffic of the machine you are interested in, PROVIDED you are on the same hub as the machine in question. Unfortunately, there's no such thing as a gigabit hub at this stage in the game.

2) Get a MANAGED switch that allows you to do port mirroring. This is what I am doing. This type of switch allows you to essentially take two of the switch ports and configure them as a 'mini hub', thus allowing me to monitor the traffic to the server in question.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #12 (permalink)  
Old 23rd February, 2005, 12:29 AM
Member/Contributor/Resident Crystal Ball
 
Join Date: March 2004
Posts: 7,451

It's not negotiating across the wrong network, is it?
__________________
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #13 (permalink)  
Old 23rd February, 2005, 04:24 PM
Gizmo's Avatar
Chief BBS Administrator
BassTeroids Champion, Global Player Champion, Aim & Fire Champion, Puzzle Maniax Champion, Othello Champion, Canyon Glider Champion, Unicycle Challenge Champion, YetiSports 9: Final Spit Champion, Zed Champion
 
Join Date: May 2003
Location: Webb City, Mo
Posts: 16,178
Send a message via ICQ to Gizmo Send a message via AIM to Gizmo Send a message via MSN to Gizmo Send a message via Yahoo to Gizmo Send a message via Skype™ to Gizmo

I can't say for 100% certain that it isn't, but given the intermittent nature of the problem, I tend to doubt it. The internal network is on a completely different subnet that is unused anywhere else in the facility, and all internal communications is done by ip address to ensure that traffic gets routed appropriately. The Computer Browser service is also disabled in order to ensure that none of the servers becomes a Master or Backup Browser.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Double NICs captaincurran Mobile Devices and Networking 5 28th January, 2006 06:38 PM
Question about SATA and Gigabit LAN QNX EPoX MotherBoards 16 15th May, 2005 04:56 AM
linux and integrated NICs barneygumble742 OS, Software, Firmware, and BIOS 12 8th March, 2005 11:54 PM
Netgear GA311 - Gigabit NIC BOD General Hardware Discussion 43 4th February, 2005 05:52 PM
Gigabit switch danrok Mobile Devices and Networking 7 8th March, 2004 02:09 PM


All times are GMT +1. The time now is 05:13 PM.


Copyright ©2001 - 2010, AOA Forums
Don't Click Here Don't Click Here Either

Search Engine Friendly URLs by vBSEO 3.3.0